Imaging apparatus comprising coding element and spectroscopic system comprising the imaging apparatus

ABSTRACT

An imaging apparatus according to an aspect of the present disclosure includes a first coding element that includes regions arrayed two-dimensionally in an optical path of light incident from an object, and an image sensor. Each of the regions includes first and second regions. A wavelength distribution of an optical transmittance of the first region has a maximum in each of first and second wavelength bands, and a wavelength distribution of an optical transmittance of the second region has a maximum in each of third and fourth wavelength bands. At least one selected from the group of the first and second wavelength bands differs from the third and fourth wavelength bands. The image sensor acquires an image in which components of the first, second, third and fourth wavelength bands of the light that has passed through the first coding element are superimposed on one another.

BACKGROUND

1. Technical Field

The present disclosure relates to coding elements, imaging apparatuses,and spectroscopic systems for acquiring spectral images and tospectroscopic methods in which such coding elements, imagingapparatuses, and spectroscopic systems are used.

2. Description of the Related Art

The use of spectral information of a number of narrow bands (e.g.,several tens of bands or more) makes it possible to grasp detailedphysical properties of an observation object, which has been impossiblewith conventional RGB images. Cameras for acquiring suchmulti-wavelength information are called hyperspectral cameras.Hyperspectral cameras are used in a variety of fields, including foodinspection, biopsy, drug development, and mineral component analyses.

As an exemplary use of images acquired with wavelengths to be observedbeing limited to narrow bands, International Publication No. WO2013/1002350 discloses an apparatus for distinguishing between a tumorsite and a non-tumor site of a subject. This apparatus detectsfluorescence at 635 nm from protoporphyrin IX accumulated in cancercells and fluorescence at 675 nm from photo-protoporphyrin that areemitted in response to irradiation of pumping light. Thus, a tumor siteand a non-tumor site are identified.

Japanese Unexamined Patent Application Publication No. 2007-108124discloses a method for determining the freshness of perishables thatdecreases with time by acquiring information on the reflectancecharacteristics of continuous multi-wavelength light.

Hyperspectral cameras that can obtain multi-wavelength images or measuremulti-wavelength reflectance can roughly be divided into the followingfour types:

-   (a) line-sensor-based hyperspectral cameras-   (b) electrofilter-based hyperspectral cameras-   (c) Fourier-transform-based hyperspectral cameras-   (d) interference-filter-based hyperspectral cameras

(a) With a line-sensor-based hyperspectral camera, one-dimensionalinformation of an object is acquired by using a member having a linearslit. Light that has passed through the slit is spot in accordance withthe wavelengths by a dispersive element, such as a diffraction gratingand a prism. The split light rays of the respective wavelengths aredetected by an image sensor having a plurality of pixels arrayedtwo-dimensionally. This method allows only one-dimensional informationof the object to be obtained at once. Thus, two-dimensional spectralinformation is obtained by scanning the entire camera or the entiremeasurement object in a direction perpendicular to the direction inwhich the slit extends. Line-sensor-based hyperspectral cameras have anadvantage that high-resolution multi-wavelength images can be obtained.Japanese Unexamined Patent Application Publication No. 2011-89895discloses an example of line-sensor-based hyperspectral cameras.

(b) An electrofilter-based hyperspectral camera that includes aliquid-crystal tunable filter (LCTF) and an electrofilter-basedhyperspectral camera that includes an acousto-optic tunable filter(AOTF) are available. A liquid-crystal tunable filter is an element inwhich a linear polarizer, a birefringent filter, and a liquid-crystalcell are arranged in multiple stages. Light at unwanted wavelengths canbe removed only by controlling the voltage, and light only at a specificdesired wavelength can be extracted. An acousto-optic tunable filter isconstituted by an acousto-optic crystal to which a piezoelectric elementis bonded. Upon an electric signal being applied to the acousto-opticcrystal, ultrasonic waves are generated, and compressional standingwaves are produced inside the crystal. Through the diffraction effect ofthe standing waves, light only at a specific desired wavelength can beextracted. This method has an advantage that high-resolution movingimage data can be obtained, although the wavelengths are limited.

(c) A Fourier-transform-based hyperspectral camera utilizes theprinciple of a two-beam interferometer. A light beam from an object tobe measured is split by a beam splitter. The respective split lightbeams are then reflected by a stationary mirror and a movable mirror,recombined together, and detected by a detector. By temporally varyingthe position of the movable mirror, data indicating a change in theinterference intensity that is dependent on the wavelength of light canbe acquired. The obtained data is subjected to the Fourier transform,and the spectral information is thus obtained. The advantage of theFourier-transform-based hyperspectral camera is that information onmultiple wavelengths can be obtained simultaneously.

(d) An interference-filter-based hyperspectral camera utilizes theprinciple of a Fabry-Perot interferometer. A configuration in which anoptical element having two surfaces with high reflectance that arespaced apart by a predetermined distance is disposed on a sensor isemployed. The distance between the two surfaces of the optical elementdiffers in different regions and is determined so as to match aninterference condition of light at a desired wavelength. Aninterference-filter-based hyperspectral camera has an advantage thatinformation on multiple wavelength can be acquired simultaneously in theform of a moving image.

Aside from the above-described methods, there is a method in whichcompressed sensing is used, as disclosed, for example, in U.S. Pat. No.7,283,231. The apparatus disclosed in U.S. Pat. No. 7,283,231 splitslight from an object to be measured by a first dispersive element, suchas a prism, marks with a coding mask, and returns the path of the lightray by a second dispersive element. Thus, an image that has been codedand multiplexed with respect to the wavelength axis is acquired by asensor. By applying the compressed sensing, a plurality of images ofmultiple wavelengths can be reconstructed from the multiplexed image.

The compressed sensing is a technique for reconstructing, from a smallnumber of samples of acquired data, a greater number of pieces of data.When the two-dimensional coordinates of an object to be measured are(x,y) and the wavelength is λ, data f to be obtained isthree-dimensional data of x, y, and λ. In the meantime, image data gobtained by the sensor is two-dimensional data that has been compressedand multiplexed in the λ-axis direction. The problem of obtaining thedata f, which has a larger amount of data, from the obtained image g,which has a smaller amount of data, is a so-called ill-posed problem andcannot be solved as-is. However, typically, data of a natural image hasredundancy, and by using the redundancy efficiently, this ill-posedproblem can be transformed to a well-posed problem. JPEG compression isan example of techniques for reducing the amount of data by utilizingthe redundancy of an image. JPEG compression employs a method in whichimage information is converted to frequency components and anonessential portion of the data, such as a component with low visualrecognizability, is removed. In the compressed sensing, such a techniqueis incorporated into an operation process, and the data space to beobtained is transformed into a space expressed by the redundancy. Thus,the unknowns are reduced, and the solution is obtained. In thistransformation, for example, the discrete cosine transform (DCT), thewavelet transform, the Fourier transform, the total variation (TV), orthe like is used.

SUMMARY

In one general aspect, the techniques disclosed here feature an imagingapparatus that includes a first coding element that includes regionsarrayed two-dimensionally in an optical path of light incident from anobject and an image sensor disposed in an optical path of light that haspassed through the first coding element. The regions include a firstregion and a second region. A wavelength distribution of an opticaltransmittance of the first region has a local maximum in each of a firstwavelength band and a second wavelength band that differ from eachother, and a wavelength distribution of an optical transmittance of thesecond region has a local maximum in each of a third wavelength band anda fourth wavelength band that differ from each other. When thewavelength distribution of the optical transmittance of the first regionis normalized such that the optical transmittance of the first regionhas a maximum value of 1 and a minimum value of 0, the local maxima inthe first wavelength band and the second wavelength band are both noless than 0.5, and when the wavelength distribution of the opticaltransmittance of the second region is normalized such that the opticaltransmittance of the second region has a maximum value of 1 and aminimum value of 0, the local maxima in the third wavelength band andthe fourth wavelength band are both no less than 0.5. At least oneselected from the group of the first wavelength band and the secondwavelength band differs from the third wavelength band and the fourthwavelength band. The image sensor acquires an Image in which componentsof the first wavelength band, the second wavelength band, the thirdwavelength band, and the fourth wavelength band of the light that haspassed through the first coding element are superimposed on one another.

According to the present disclosure, an occurrence of coma aberrationand a decrease in the resolution associated with the occurrence of comaaberration can be suppressed.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A schematically illustrates a configuration of a coding element Caccording to a first embodiment;

FIG. 1B illustrates an example of the spatial distribution of thetransmittance of light in each of a plurality of wavelength bands W1,W2, . . . , and Wi included in a target wavelength band of the codingelement C according to the first embodiment;

FIG. 1C illustrates an example of the spectral transmittance in a regionA1 of the coding element C according to the first embodiment;

FIG. 1D illustrates an example of the spectral transmittance in a regionA2 of the coding element C according to the first embodiment;

FIG. 2A is an illustration for describing a relation between the targetwavelength band W and the plurality of wavelength bands W1, W2, . . . ,and Wi included in the target wavelength band W;

FIG. 2B illustrates an example in which the band widths vary among thewavelength bands and a gap is present between two adjacent wavelengthbands;

FIG. 3A is an illustration for describing the characteristics of thespectral transmittance in a given region of the coding element C;

FIG. 3B illustrates a result of averaging the spectral transmittanceillustrated in FIG. 3A in each of the wavelength bands W1, W2, . . . ,and Wi;

FIG. 4 is a schematic diagram illustrating an imaging apparatus D1according to a second embodiment of the present disclosure;

FIG. 5A illustrates an example of the transmittance distribution of thecoding element C according to the present embodiment;

FIG. 5B illustrates another configuration example of the coding elementC;

FIG. 6A illustrates, in approximation, the spectral transmittance in agiven region of the coding element C having the binary-scaletransmittance distribution illustrated in FIG. 5A;

FIG. 6B illustrates an example of the spectral transmittance in a givenregion of the coding element C having the binary-scale transmittancedistribution illustrated in FIG. 5A;

FIG. 7 is a flowchart illustrating an overview of a spectroscopic methodaccording to the second embodiment;

FIG. 8A is a schematic diagram illustrating an imaging apparatus D2according to a third embodiment of the present disclosure;

FIG. 8B is a schematic diagram illustrating an imaging apparatus D2′according to a modification of the third embodiment of the presentdisclosure;

FIG. 9 illustrates an example of a result of reconstructing spectrallyseparated images F in a first implementation example of the presentdisclosure;

FIG. 10 illustrates an example of a result of reconstructing spectrallyseparated images F in a first comparative example;

FIG. 11 illustrates a mean squared error (MSE) relative to a correctimage in each of the first implementation example and the firstcomparative example;

FIG. 12 is a schematic diagram illustrating a spectroscopic system S1according to a fourth embodiment of the present disclosure;

FIG. 13A illustrates an example of the spectral transmittance of anarrow-band coding element in an imaging apparatus D4 illustrated inFIG. 12;

FIG. 13B illustrates another example of the spectral transmittance ofthe narrow-band coding element in the imaging apparatus D4 illustratedin FIG. 12;

FIG. 13C illustrates yet another example of the spectral transmittanceof the narrow-band coding element;

FIG. 13D illustrates yet another example of the spectral transmittanceof the narrow-band coding element;

FIG. 13E illustrates yet another example of the spectral transmittanceof the narrow-band coding element;

FIG. 14A is an illustration for describing a relation between a targetwavelength band W and a plurality of wavelength bands W1, W2, . . . ,and Wn included in the target wavelength band W;

FIG. 14B illustrates an example in which the band widths vary among thewavelength bands and a gap is present between two adjacent wavelengthbands;

FIG. 15 illustrates another configuration example of the imagingapparatus D4 in the spectroscopic system S1;

FIG. 16A schematically illustrates a configuration of a spatiallymodulating coding element CS of the imaging apparatus D4 illustrated inFIG. 12;

FIG. 16B illustrates an example of the spatial distribution of thetransmittance of light in each of the plurality of wavelength bands W1,W2, . . . , and Wi included in the target wavelength band of thespatially modulating coding element CS of the imaging apparatus D4illustrated in FIG. 12;

FIG. 16C illustrates an example of the spectral transmittance of in aregion A1 of the spatially modulating coding element CS of the imagingapparatus D4 illustrated in FIG. 12;

FIG. 16D illustrates an example of the spectral transmittance of in aregion A2 of the spatially modulating coding element CS of the imagingapparatus D4 illustrated in FIG. 12;

FIG. 17 is an illustration for describing the wavelength resolving powerof the spectral transmittance in a given region of the spatiallymodulating coding element CS;

FIG. 18A illustrates an example of the wavelength and the intensity oflight obtained when the light has passed through a narrow-band codingelement C1;

FIG. 18B illustrates an example of the wavelength and the intensity oflight obtained when the light has passed through the spatiallymodulating coding element CS;

FIG. 18C illustrates an example of the wavelength and the intensity oflight obtained when the light has passed through both the narrow-bandcoding element C1 and the spatially modulating coding element CS;

FIG. 19 illustrates an example of the wavelength and the intensity oflight obtained when the light has passed through both the narrow-bandcoding element C1 and the spatially modulating coding element CS;

FIG. 20A illustrates an example of a binary-scale spectraltransmittance;

FIG. 20B illustrates another example of a binary-scale spectraltransmittance;

FIG. 21A illustrates an example of the transmittance distribution of thespatially modulating coding element CS according to the fourthembodiment;

FIG. 21B illustrates another example of the transmittance distributionof the spatially modulating coding element CS according to the fourthembodiment; and

FIG. 22 is a flowchart illustrating an overview of a spectroscopicmethod according to the fourth embodiment.

DETAILED DESCRIPTION

Prior to describing embodiments of the present disclosure, underlyingknowledge found by the present inventor will be described.

According to the study by the present inventor, the conventionalhyperspectral cameras described above have the following issues. (a) Theline-sensor-based hyperspectral camera needs to be scanned in order toobtain a two-dimensional image and is thus not suitable for capturing amoving image of an object to be measured. (c) With theFourier-transform-based hyperspectral camera as well, the reflectivemirror needs to be moved, and the Fourier-transform-based hyperspectralcamera is thus not suitable for capturing a moving image. (b) With theelectrofilter-based hyperspectral camera, an image is acquired at eachwavelength, and a multi-wavelength image cannot be obtained at once. (d)With the interference-filter-based hyperspectral camera, there is atrade-off between the number of wavelength bands in which images can beacquired and the spatial resolving power, and thus the spatial resolvingpower is compromised when a multi-wavelength image is acquired. In thismanner, none of the existing hyperspectral cameras simultaneouslysatisfy the three conditions: a high resolution, multiple wavelengths,and moving image capturing (one-shot shooting).

The configuration that uses the compressed sensing appears tosimultaneously satisfy the three conditions: a high-resolution, multiplewavelengths, and moving image capturing. However, an image isreconstructed on the basis of estimation from a small amount of data,and thus the spatial resolution of the acquired image is likely to belower than that of the original image. In particular, as the compressionrate of the acquired data is higher, effect thereof appears moreprominently. Furthermore, since a dispersive element, such as a prism,is disposed in an optical path, coma aberration occurs, which leads to aproblem in that the resolution decreases.

The present inventor has found the above-described problems and examinedconfigurations for solving these problems. The present inventor hasfound that an occurrence of coma aberration can be suppressed and theresolution can be increased by appropriately designing the spectraltransmittance in each region of a coding element. According to anembodiment of the present disclosure, the three demands, namely, a highresolution, multiple wavelengths, and moving image capturing (one-shotshooting) can be fulfilled simultaneously. In addition, in an embodimentof the present disclosure, of the three-dimensional information of thex-direction, the y-direction, and the wavelength direction, theinformation in the wavelength direction is compressed. Therefore, onlythe two-dimensional data needs to be retained, and the amount of datacan be reduced. Thus, an embodiment of the present disclosure iseffective when data of an extended period of time is to be acquired.

The present disclosure includes an imaging apparatus, a system, and amethod according to the following items.

[Item 1]

An imaging apparatus, comprising

a first coding element that includes regions arrayed two-dimensionallyin an optical path of light incident from an object; and

an image sensor disposed in an optical path of light that has passedthrough the first coding element,

wherein the regions include a first region and a second region,

wherein a wavelength distribution of an optical transmittance of thefirst region has a local maximum in each of a first wavelength band anda second wavelength band that differ from each other,

wherein a wavelength distribution of an optical transmittance of thesecond region has a local maximum in each of a third wavelength band anda fourth wavelength band that differ from each other,

wherein, when the wavelength distribution of the optical transmittanceof the first region is normalized such that the optical transmittance ofthe first region has a maximum value of 1 and a minimum value of 0, thelocal maxima in the first wavelength band and the second wavelength bandare both no less than 0.5,

wherein, when the wavelength distribution of the optical transmittanceof the second region is normalized such that the optical transmittanceof the second region has a maximum value of 1 and a minimum value of 0,the local maxima in the third wavelength band and the fourth wavelengthband are both no less than 0.5,

wherein, at least one selected from the group of the first wavelengthband and the second wavelength band differs from the third wavelengthband and the fourth wavelength band, and

wherein, in operation, the image sensor acquires an image in whichcomponents of the first wavelength band, the second wavelength band, thethird wavelength band, and the fourth wavelength band of the light thathas passed through the first coding element are superimposed on oneanother.

[Item 2]

The imaging apparatus according to Item 1, wherein the regions includeat least one transparent region.

[Item 3]

The imaging apparatus according to Item 2, wherein the at least onetransparent region comprises a plurality of transparent regions, whereinthe regions include regions whose optical transmittance differs atdifferent wavelengths and the plurality of transparent regions, the tworegions are arrayed in an alternating manner in one array direction ofthe regions and another array direction that is perpendicular to the onearray direction.

[Item 4]

The imaging apparatus according to any one of Items 1 to 3, wherein theregions are arrayed two-dimensionally in a matrix,

wherein a vector having, as its elements, values of transmittance oflight in a fifth wavelength band in respective regions belonging to aset of regions arrayed in a single row or column included in the regionsand a vector having, as its elements, values of transmittance of lightin the fifth wavelength band in respective regions belonging to a set ofregions arrayed in another row or column included in the regions areindependent from each other, and

wherein a vector having, as its elements, values of transmittance oflight in a sixth wavelength band in respective regions belonging to aset of regions arrayed in a single row or column included in the regionsand a vector having, as its elements, values of transmittance of lightin the sixth wavelength band in respective regions belonging to a set ofregions arrayed in another row or column included in the regions areindependent from each other.

[Item 5]

The imaging apparatus according to any one of Items 1 to 4, furthercomprising an optical system that is disposed between the object and thefirst coding element and that converges the light from the object on asurface of the first coding element,

wherein the first coding element is disposed on the image sensor.

[Item 6]

The imaging apparatus according to Item 5, wherein the image sensorincludes pixels, and

wherein the regions correspond to the respective pixels.

[Item 7]

The imaging apparatus according to any one of Items 1 to 4, furthercomprising an optical system that is disposed between the object and thefirst coding element and that converges the light from the object on asurface of the image sensor,

wherein the first coding element and the image sensor are spaced apartfrom each other.

[Item 8]

The imaging apparatus according to any one of Items 1 to 4, furthercomprising an optical system that is disposed between the first codingelement and the image sensor and that converges the light from theobject that has passed through the first coding element on a surface ofthe image sensor.

[Item 9]

The imaging apparatus according to any one of Items 1 to 8, furthercomprising a signal processing circuit that, in operation, generatesimages in respective wavelength bands of the light that has passedthrough the first coding element on the basis of the image acquired bythe image sensor and a spatial distribution and a wavelengthdistribution of an optical transmittance of the first coding element.

[Item 10]

The imaging apparatus according to Item 9, wherein, in operation, thesignal processing circuit generates the images in the respectivewavelength bands through a statistical method.

[Item 11]

The imaging apparatus according to Item 9 or 10, wherein the number ofpieces of data in the images in the respective wavelength bands isgreater than the number of pieces of data in the image acquired by theimage sensor.

[Item 12]

The imaging apparatus according to any one of Items 9 to 11, wherein theimage sensor includes pixels, and

wherein, in operation, the signal processing circuit generates, as theimages in the respective wavelength bands, a vector f′ estimated on thebasis of the expression

${f^{\prime} = {\underset{f}{\arg\;\min}\left\{ {{{g - {Hf}}}_{l_{2}} + {{\tau\Phi}(f)}} \right\}}},$wherein φ(f) is a regularization term and τ is a weighting factor, byusing a vector g having, as its elements, signal values of the pixels inthe image acquired by the image sensor and a matrix H determined by thespatial distribution and the wavelength distribution of the opticaltransmittance of the first coding element.

[Item 13]

The imaging apparatus according to any one of Items 9 to 12, wherein, inoperation, the signal processing circuit generates the images in therespective wavelength bands in the form of a moving image.

[Item 14]

The imaging apparatus according to any one of Items 1 to 13, furthercomprising at least one second coding element whose opticaltransmittance is uniform in a spatial direction and that includeslight-transmitting regions and light-blocking regions arrayed in thewavelength direction,

wherein the image sensor is disposed in an optical path of light thathas passed through the first coding element and the at least one secondcoding element.

[Item 15]

The imaging apparatus according to Item 14, wherein, in the at least onesecond coding element, the light-transmitting regions have an equalwavelength band width and the light-blocking regions present between twoclosest light-transmitting regions in the light-transmitting regionshave an equal wavelength band width.

[Item 16]

The imaging apparatus according to Item 14 or 15, wherein the at leastone second coding element comprises a plurality of second codingelements, and

wherein wavelength bands of the light-transmitting regions in one of theplurality of second coding elements are different from wavelength bandsof the light-transmitting regions in another one of the plurality ofsecond coding elements.

[Item 17]

The imaging apparatus according to any one of Items 14 to 16, furthercomprising a signal processing circuit that, in operation, generatesimages in respective wavelength bands of the light that has passedthrough the first coding element and the at least one second codingelement on the basis of the image output by the image sensor, a spatialdistribution and a wavelength distribution of an optical transmittanceof the first coding element, and a wavelength distribution of an opticaltransmittance of the at least one second coding element.

[Item 18]

The imaging apparatus according to any one of Items 1 to 17, wherein thewavelength distribution of the optical transmittance in each of theregions is a random distribution.

[Item 19]

The imaging apparatus according to any one of Items 1 to 18, wherein aspatial distribution of the optical transmittance of the first codingelement in each of the first wavelength band, the second wavelengthband, the third wavelength band, and the fourth wavelength band is arandom distribution.

[Item 20]

A spectroscopic system comprising:

an imaging apparatus that includes

-   -   a first coding element that includes regions arrayed        two-dimensionally in an optical path of light incident from an        object; and    -   an image sensor disposed in an optical path of light that has        passed through the first coding element,    -   wherein the regions include a first region and a second region,    -   wherein a wavelength distribution of an optical transmittance of        the first region has a local maximum in each of a first        wavelength band and a second wavelength band that differ from        each other,    -   wherein a wavelength distribution of an optical transmittance of        the second region has a local maximum in each of a third        wavelength band and a fourth wavelength band that differ from        each other,    -   wherein, when the wavelength distribution of the optical        transmittance of the first region is normalized such that the        optical transmittance of the first region has a maximum value of        1 and a minimum value of 0, the local maxima in the first        wavelength band and the second wavelength band are both no less        than 0.5,    -   wherein, when the wavelength distribution of the optical        transmittance of the second region is normalized such that the        optical transmittance of the second region has a maximum value        of 1 and a minimum value of 0, the local maxima in the third        wavelength band and the fourth wavelength band are both no less        than 0.5,    -   wherein, at least one selected from the group of the first        wavelength band and the second wavelength band differs from the        third wavelength band and the fourth wavelength band, and    -   wherein, in operation, the image sensor acquires an image in        which components of the first wavelength band, the second        wavelength band, the third wavelength band, and the fourth        wavelength band of light that has passed through the first        coding element are superimposed on one another; and

a signal processing device that, in operation, generates images inrespective wavelength bands of the light that has passed through thefirst coding element on the basis of the image acquired by the imagesensor and a spatial distribution and a wavelength distribution of anoptical transmittance of the first coding element.

[Item 21]

A coding element to be used in a spectroscopic system that generates animage in each of mutually different wavelength bands, the coding elementcomprising:

regions arrayed two-dimensionally, wherein the regions include two ormore regions having mutually different spectral transmittances, and thespectral transmittance in each of the two or more regions has localmaxima in at least two of the wavelength bands.

[Item 22]

The coding element according to Item 21, wherein the opticaltransmittance at the local maxima is no less than 0.8.

[Item 23]

The coding element according to Item 21 or 22, wherein combinations ofthe at least two wavelength bands in the two or more regions differ fromeach other.

[Item 24]

The coding element according to any one of Items 21 to 23, wherein theregions include at least one transparent region,

[Item 25]

The coding element according to Item 24, wherein the at least onetransparent region comprises a plurality of transparent regions,

wherein the regions include regions whose optical transmittance differsin difference wavelengths and the plurality of transparent regions, thetwo regions are arrayed in an alternating manner in two array directionsof the regions,

[Item 26]

A coding element to be used in a spectroscopic system that generates animage in each of wavelength bands including an image in a firstwavelength band and an image in a second wavelength band, the codingelement comprising:

regions that are arrayed two-dimensionally in a matrix,

wherein a vector having, as its elements, values of transmittance oflight in the first wavelength band in respective regions belonging to aset of regions arrayed in a single row or column included in the regionsand a vector having, as its elements, values of transmittance of lightin the first wavelength band in respective regions belonging to a set ofregions arrayed in another row or column included in the regions areindependent from each other,

wherein a vector having, as its elements, values of transmittance oflight in the second wavelength band in respective regions belonging to aset of regions arrayed in a single row or column included in the regionsand a vector having, as its elements, values of transmittance of lightin the second wavelength band in respective regions belonging to a setof regions arrayed in another row or column included in the regions areindependent from each other, and

wherein a spectral transmittance in each of two or more regions includedin the regions has local maxima in the first and second wavelengthbands.

[Item 27]

An imaging apparatus, comprising:

the coding element according to any one of Items 21 to 26 that isdisposed in an optical path of light incident from an object; and

an image sensor that is disposed in an optical path of light that haspassed through the coding element and that acquires an image in whichcomponents of the wavelength bands that have passed through the codingelement are superimposed on one another.

[Item 28]

The imaging apparatus according to Item 27, further comprising anoptical system that is disposed between the object and the codingelement and that converges the light from the object on a surface of thecoding element,

wherein the coding element is disposed on the image sensor.

[Item 29]

The imaging apparatus according to Item 28, wherein the regions in thecoding element correspond to respective pixels in the image sensor.

[Item 30]

The imaging apparatus according to Item 27, further comprising anoptical system that is disposed between the object and the codingelement and that converges the light from the object on a surface of theimage sensor,

wherein the coding element and the image sensor are spaced apart fromeach other.

[Item 31]

The imaging apparatus according to Item 27, further comprising anoptical system that is disposed between the coding element and the imagesensor and that converges the light from the object that has passedthrough the coding element on a surface of the image sensor.

[Item 32]

The imaging apparatus according to any one of Items 27 to 31, furthercomprising a signal processing circuit that, in operation, generatesimages in respective wavelength bands of the light that has passedthrough the coding element on the basis of the image acquired by theimage sensor and a spatial distribution of a spectral transmittance ofthe coding element.

[Item 33]

The imaging apparatus according to Item 32, wherein the signalprocessing circuit generates the images in the respective wavelengthbands through a statistical method.

[Item 34]

The imaging apparatus according to Item 32 or 33, wherein the number ofpieces of data of the images in the respective wavelength bands of thelight is greater than the number of pieces of data of the image acquiredby the image sensor.

[Item 35]

The imaging apparatus according to any one of Items 32 to 34, wherein,in operation, the signal processing circuit generates, as the images inthe respective wavelength bands, a vector f′ estimated on the basis ofthe expression

${f^{\prime} = {\underset{f}{\arg\;\min}\left\{ {{{g - {Hf}}}_{l_{2}} + {{\tau\Phi}(f)}} \right\}}},$wherein φ(f) is a regularization term and τ is a weighting factor, byusing a vector g having, as its elements, signal values of the pixels inthe image acquired by the image sensor and a matrix H determined by thespatial distribution of the spectral transmittance of the codingelement.

[Item 36]

The imaging apparatus according to any one of Items 32 to 35, whereinthe signal processing circuit generates the images in the respectivewavelength bands in the form of a moving image.

[Item 37]

A spectroscopic system, comprising:

the imaging apparatus according to any one of Items 27 to 31; and

a signal processing device that, in operation, generates images inrespective wavelength bands of the light that has passed through thecoding element on the basis of an image acquired by the image sensor anda spatial distribution of a spectral transmittance of the codingelement.

[Item 38]

A spectroscopic method, comprising:

modulating an intensity of incident light by using the coding elementaccording to any one of Items 21 to 26 that is disposed in an opticalpath of light incident from an object;

acquiring an image in which components of wavelength bands of light thathas passed through the coding element are superimposed on one another;and

generating images in respective wavelength bands of the light that haspassed through the coding element on the basis of the image and aspatial distribution of a spectral transmittance of the coding element.

[Item 41]

An imaging apparatus to be used in a spectroscopic system that generatesimages in respectively different wavelength bands, the imaging apparatuscomprising:

at least one narrow-band coding element whose optical transmittance isuniform in a spatial direction and that includes light-transmittingregions and light-blocking regions arrayed in a wavelength direction;

a spatially modulating coding element that is disposed in a path oflight that passes through the at least one narrow-band coding elementand that includes light-transmitting regions and light-blocking regionsarrayed in the spatial direction; and

an image sensor that, in operation, acquires light coded by the at leastone narrow-band coding element and the spatially modulating codingelement.

[Item 42]

The imaging apparatus according to Item 41, wherein, in the at least onenarrow-band coding element, the light-transmitting regions have an equalwavelength band width and the light-blocking regions between two closestlight-transmitting regions have an equal wavelength band width.

[Item 43]

The imaging apparatus according to Item 41 or 42, wherein the at leastone narrow-band coding element comprises a plurality of narrow-bandcoding elements, and the light-transmitting regions in the plurality ofnarrow-band coding elements are mutually different wavelength ranges.

[Item 44]

The imaging apparatus according to Item 43, wherein thelight-transmitting regions in the plurality of narrow-band codingelements include all of the different wavelength bands.

[Item 45]

The imaging apparatus according to any one of Items 41 to 44, wherein,in the spatially modulating coding element, the spatial distribution ofthe light-transmitting regions and the light-blocking regions differs inthe different wavelength bands.

[Item 46]

The imaging apparatus according to any one of Items 41 to 44, furthercomprising a dispersive element that is disposed in the path of thelight that passes through the at least one narrow-band coding elementand that disperses light in the spatial direction in accordance with thewavelength, wherein the optical transmittance of the spatiallymodulating coding element is uniform in the wavelength direction.

[Item 47]

A spectroscopic system, comprising:

the imaging apparatus according to Item 45; and

a signal processing circuit that, in operation, generates images inrespectively different wavelength bands on the basis of a captured imageoutput from the image sensor in the imaging apparatus, wavelengthdistribution information of the optical transmittance of the at leastone narrow-band coding element, and spatial distribution information andwavelength distribution information of the optical transmittance of thespatially modulating coding element.

[Item 48]

A spectroscopic system, comprising:

the imaging apparatus according to Item 46; and

a signal processing circuit that, in operation, generates images inrespectively different wavelength bands on the basis of a captured imageoutput from the image sensor in the imaging apparatus, wavelengthdistribution information of the optical transmittance of the at leastone narrow-band coding element, spatial distribution information of theoptical transmittance of the spatially modulating coding element, anddispersion characteristics of the dispersive element.

[Item 49]

A spectral filter to be used in a spectroscopic system that, inoperation, generates images in respectively different wavelength bands,the spectral filter comprising:

light-transmitting regions and light-blocking regions arrayed in aspatial direction, wherein the light-transmitting regions have an equalwavelength band width and the light-blocking regions present between twoclosest light-transmitting regions have an equal wavelength band width.

[Item 50]

A spectroscopic method in which used is an imaging apparatus thatincludes a first narrow-band coding element and a second narrow-bandcoding element each having an optical transmittance that is uniform in aspatial direction and each including light-transmitting regions andlight-blocking regions arrayed in a wavelength direction, a spatiallymodulating coding element that is disposed in a path of light thatpasses through one of the first narrow-band coding element and thesecond narrow-band coding element and that includes light-transmittingregions and light-blocking regions arrayed in the spatial direction, andan image sensor that, in operation, acquires light coded by the firstnarrow-band coding element, the second narrow-band coding element, andthe spatially modulating coding element,

wherein, in operation, the first narrow-band coding element and thespatially modulating coding element code light from an object, the imagesensor acquires light coded by the first narrow-band coding element andthe spatially modulating coding element to generate a first pixelsignal, the first narrow-band coding element is replaced with the secondnarrow-band coding element, the second narrow-band coding element andthe spatially modulating coding element code the light from the object,and the image sensor acquires light coded by the second narrow-bandcoding element and the spatially modulating coding element to generate asecond pixel signal.

Hereinafter, more specific embodiments of the present disclosure will bedescribed with reference to the drawings. In the following description,a signal indicating an image (a set of signals representing the pixelvalues of respective pixels) may simply be referred to as an image. Inthe following description, the xyz-coordinates indicated in the drawingswill be used.

First Embodiment

FIGS. 1A to 1D are illustrations for describing a coding element Caccording to a first embodiment. The coding element C is used in aspectroscopic system that generates an image in each of a plurality ofwavelength bands included in a wavelength band to be imaged. In thepresent specification, the wavelength band to be imaged may be referredto as a target wavelength band. The coding element C is disposed in anoptical path of light from an object. The coding element C modulates theintensity of incident light at each wavelength and outputs the result.This process of the coding element C is referred to as coding in thepresent specification. The coding element C corresponds to a firstcoding element according to the present disclosure.

FIG. 1A schematically illustrates the configuration of the codingelement C. The coding element C includes a plurality of regions arrayedtwo-dimensionally. Each of the regions is formed of a member having alight-transmitting property and has an individually set spectraltransmittance. Here, the spectral transmittance means a wavelengthdistribution of an optical transmittance. The spectral transmittance isexpressed by a function T(λ), in which λ represents the wavelength ofincident light. The spectral transmittance T(λ) can take a value no lessthan 0 and no greater than 1. Although FIG. 1A illustrates 48rectangular regions arrayed in six rows by eight columns, in an actualuse, a greater number of regions can be provided. The number of regionscan, for example, be equivalent to the number of pixels (e.g., severalhundred thousand to several ten million) of a typical image sensor. Inone example, the coding element C can be disposed immediately above animage sensor such that each region corresponds to (opposes) one of thepixels in the image sensor.

FIG. 1B illustrates an example of the spatial distribution of thetransmittance of light in each of a plurality of wavelength bands W1,W2, . . . , and Wi included in a target wavelength band. In FIG. 1B, thedifference in shade among the regions (cells) represents the differencein the transmittance. A lighter region has a higher transmittance, and adarker region has a lower transmittance. As illustrated in FIG. 1B, thespatial distribution of the optical transmittance differs in differentwavelength bands.

FIGS. 1C and 1D are graphs illustrating examples of the spectraltransmittance in two regions A1 and A2, respectively, in the codingelement C. In each graph, the horizontal axis represents the wavelength,and the vertical axis represents the optical transmittance. The spectraltransmittance is normalized such that the optical transmittance in eachregion has a maximum value of 1 and a minimum value of 0. The spectraltransmittance in the region A1 differs from the spectral transmittancein the region A2. In this manner, the spectral transmittance in thecoding element C differs in different regions. However, it is notnecessary that every region have a different spectral transmittance. Itis sufficient that at least some (two or more) of the plurality ofregions in the coding element C have mutually different spectraltransmittances. In one example, the number of patterns of the spectraltransmittances in the plurality of regions included in the codingelement C can be equal to or greater than the number i of the wavelengthbands included in the target wavelength band. Typically, the codingelement C is designed such that more than half of the regions havemutually different spectral transmittances.

FIG. 2A is an illustration for describing a relation between the targetwavelength band W and the plurality of wavelength bands W1, W2, . . . ,and Wi included in the target wavelength band W. The target wavelengthband W can be set to a variety of ranges in accordance with the intendeduse. The target wavelength band W can, for example, be a visible-lightwavelength band (approximately 400 nm to approximately 700 nm), anear-infrared wavelength band (approximately 700 nm to approximately2500 nm), a near-ultraviolet wavelength band (approximately 10 nm toapproximately 400 nm), a mid-infrared band, a far-infrared band, or aradio-wave band including terahertz waves and millimeter waves. In thismanner, a wavelength band to be used is not limited to a visible-lightband. In the present specification, aside from visible light,non-visible rays including near-ultraviolet rays, near-Infrared rays,and radio waves are also referred to as light for convenience.

In the present embodiment, as illustrated in FIG. 2A, provided that i isany integer no less than 4, wavelength bands obtained by equallydividing the target wavelength band W by i are designated as thewavelength bands W1, W2, . . . , and Wi. The present embodiment,however, is not limited to this example. The plurality of wavelengthbands included in the target wavelength band W may be set as desired.For example, the widths of the wavelength bands (referred to as bandwidths) may vary among the wavelength bands. A gap may be presentbetween adjacent wavelength bands. FIG. 2B illustrates an example inwhich the band widths vary among the wavelength bands and a gap ispresent between two adjacent wavelength bands. In this manner, theplurality of wavelength bands may mutually differ and can be determinedas desired. The number i into which the wavelength band is divided maybe 3 or less.

FIG. 3A is an illustration for describing the characteristics of thespectral transmittance in a given region of the coding element C. Thespectral transmittance in this example has a plurality of local maximaP1 to P5 and a plurality of local minima with respect to the wavelengthswithin the target wavelength band W. The wavelength distribution of theoptical transmittance illustrated in FIG. 3A is normalized such that theoptical transmittance within the target wavelength band W has a maximumvalue of 1 and a minimum value of 0. In this example, the spectraltransmittance has local maxima in the wavelength bands W2, Wi-1, and soon. In this manner, in the present embodiment, the spectraltransmittance in each region has local maxima in a plurality of (at easttwo) wavelength bands among the plurality of wavelength bands W1 to Wi.As can be seen from FIG. 3A, the local maxima P1, P3, P4, and P5 are noless than 0.5.

As described thus far, the optical transmittance of each region variesdepending on the wavelength. Therefore, of incident light, the codingelement C transmits components in certain wavelength bands in a largeamount and does not transmit components in other wavelength bands in alarge amount. For example, light in k wavelength bands (k is an integerthat satisfies 2≦k<i) among the i wavelength bands has a transmittanceof greater than 0.5 (50%), and light in the remaining i-k wavelengthbands has a transmittance of less than 0,5 (50%). If the incident lightis white light that equally includes the wavelength components of theentire visible light, the coding element C modulates the incident lighthi each region into light that has a plurality of intensity peaks thatare discrete with respect to the wavelengths, superimposes the obtainedlight of multiple wavelengths, and outputs the result.

FIG. 3B illustrates, as an example, a result of averaging the spectraltransmittance illustrated in FIG. 3A in each of the wavelength bands W1,W2, . . . , and Wi. The averaged transmittance is obtained byintegrating the spectral transmittance T(λ) in each wavelength band andby dividing the result by the width of the corresponding wavelength band(band width). In the present specification, the value of thetransmittance averaged in each wavelength band in this manner isreferred to as the transmittance in the corresponding wavelength band.In this example, the transmittance is saliently high in the threewavelength bands having the local maxima P1, P3, and P5. In particular,the transmittance exceeds 0.8 (80%) in the wavelength bands having thelocal maxima P3 and P5.

The resolving power of the spectral transmittance in each region in thewavelength direction can be set approximately to a desired wavelengthband width (band width). In other words, in a wavelength range thatincludes one local maximum (peak) in the spectral transmittance curve,the width of the range that takes a value no less than a mean value of alocal minimum closest to the stated local maximum and the stated localmaximum can be set approximately to a desired wavelength band width(band width). In this case, if the spectral transmittance is resolvedinto frequency components by using the Fourier transform or the like,the value of a frequency component corresponding to the statedwavelength band becomes relatively large.

As illustrated in FIG. 1A, typically, the coding element C is dividedinto a plurality of regions (cells) of a lattice pattern. These cellshave mutually different spectral transmittance characteristics. Thewavelength distribution and the spatial distribution of the opticaltransmittance in each region of the coding element C can, for example,be a random distribution or a quasi-random distribution.

The random distribution and the quasi-random distribution are consideredas follows. First, each region of the coding element C can be consideredas a vector element having a value of, for example, 0 to 1 in accordancewith the optical transmittance. Here, when the transmittance is 0 (0%),the value of the vector element is 0; and if the transmittance is 1(100%), the value of the vector element is 1. In other words, a set ofregions arrayed in a line in a row direction or a column direction canbe considered as a multi-dimensional vector having a value of 0 to 1.Therefore, it can be said that the coding element C includes a pluralityof multi-dimensional vectors in the row direction or the columndirection. In this case, the random distribution means that any twomulti-dimensional vectors are independent from each other (they are notparallel). Meanwhile, the quasi-random distribution means that aconfiguration in which multi-dimensional vectors are not independent isincluded in some of the multi-dimensional vectors. Therefore, in therandom distribution and the quasi-random distribution, a vector having,as its elements, the values of the transmittance of light in a firstwavelength band in respective regions belonging to a set of regionsarrayed in a single row (or column) included in the plurality of regionsis independent from a vector having, as its elements, the values of thetransmittance of light in the first wavelength band in respectiveregions belonging to another set of regions arrayed in a row (orcolumn). In a similar manner, with respect to a second wavelength bandthat differs from the first wavelength band, a vector having, as anelement, the value of the transmittance of light in the secondwavelength band in each region belonging to a set of regions arrayed ina single row (or column) included in the plurality of regions isindependent from a vector having, as an element, the value of thetransmittance of light in the second wavelength band in each regionbelonging to another set of regions arrayed in a row (or column).

The random distribution may be defined by an autocorrelation functiondefined by the following expression (1).

$\begin{matrix}{{y\left( {i,j,k} \right)} = {\sum\limits_{l = 1}^{L}{\sum\limits_{m = 1}^{M}{\sum\limits_{n = 1}^{N}{{x\left( {l,m,n} \right)} \cdot {x\left( {{l + i},{m + j},{n + k}} \right)}}}}}} & (1)\end{matrix}$

Provided that the coding element C is formed of a total of M×Nrectangular regions arrayed in a matrix of M in the longitudinaldirection by N in the lateral direction and that the number of spectralimages generated by a spectroscopic system that includes the codingelement C is L, x(l,m,n) represents the optical transmittance in the lthwavelength band of a rectangular region that is disposed at the mthposition in the longitudinal direction and the nth position in thelateral direction. In addition, i=−(L−1), . . . , −1, 0, 1, . . . , and(L−1); j=−(M−1), . . . , −1, 0, 1, . . . , and (M−1); and k=−(N−1), . .. , −1, 0, 1, . . . , (N−1). When m<1, n<1, l<1, m>M, n>N, and l>L;x(l,m,n)=0. The autocorrelation function y(i,j,k) indicated by the aboveexpression (1) is a function that expresses, with i, j, and k beingvariables, a correlation value between the optical transmittancex(l,m,n) in the lth wavelength band in a rectangular region disposed atthe mth position in the longitudinal direction and the nth position inthe lateral direction and the optical transmittance x(l+i,m+j,n+k) in awavelength band offset by i from the lth wavelength band in arectangular region that is offset by j in the longitudinal direction andk in the lateral direction from the aforementioned rectangular region.In this case, the random distribution as used in the present disclosuremeans, for example, that the autocorrelation function y(i,j,k) indicatedby the above expression (1) has a local maximum at y(0,0,0) and does nothave a local maximum at the other coordinates. Specifically, it meansthat the autocorrelation function y(i,j,k) monotonously decreases as ichanges from i=0 to (L−1) and to −(L−1), monotonously decreases as jchanges from j=0 to (M−1) and to −(M−1), and monotonously decreases as kchanges from k=0 to (N−1) and to −(N−1). In addition, the randomdistribution may have, aside from the local maximum at y(0,0,0), localmaxima at no greater than L/10 positions in the i-axis direction, localmaxima at no greater than M/10 positions in the j-axis direction, andlocal maxima at no greater than N/10 positions in the k-axis direction.

When the coding element C is disposed in the vicinity of or immediatelyabove the image sensor, the interval (cell pitch) among the plurality ofregions in the coding element C may substantially match the pixel pitchof the image sensor. With this configuration, the resolution of an imageof coded light emitted from the coding element C substantially matchesthe resolution of the pixels. By allowing light that has passed througheach cell to be incident only on a single corresponding pixel, theoperation described later can be simplified. If the coding element C isdisposed so as to be spaced apart from the image sensor, the cell pitchmay be reduced in accordance with the distance therebetween.

In the example illustrated in FIGS. 1A to 1D, a gray-scale transmittancedistribution in which the transmittance in each region can take anyvalue that is no less than 0 and no greater than 1 is assumed. However,an embodiment is not limited to the gray-scale transmittancedistribution. For example, as in a second embodiment described later, abinary-scale transmittance distribution in which the transmittance ineach region can take a value of either substantially 0 or substantially1 may be employed. With the binary-scale transmittance distribution,each region transmits a large portion of light in at least twowavelength bands among the plurality of wavelength bands included in thetarget wavelength band and does not transmit (blocks) a large portion oflight in the remaining wavelength bands. Here, a large portioncorresponds to approximately 80% or more.

Some (e.g., half) of the entire cells may be replaced with transparentregions. Such transparent regions transmit light in the entirewavelength bands W1 to Wi included in the target wavelength bandsubstantially equally at a high transmittance (e.g., 0.8 or more). Insuch a configuration, the plurality of transparent regions can bedisposed, for example, in a checkered pattern. In other words, in eachof the two array directions (the lateral direction and the longitudinaldirection in FIG. 1A) of the plurality of regions in the coding elementC, regions whose optical transmittances vary depending on the wavelengthand the transparent regions are arrayed in an alternating manner.

The coding element C can be constituted by at least one selected fromthe group consisting of a multilayer film, an organic material, adiffraction grating structure, and a microstructure containing metal. Ina case in which a multilayer film is used, for example, a dielectricmultilayer film or a multilayer film that includes a metal layer can beused. In this case, the cells are formed such that at least one selectedfrom the group consisting of the thickness of the multilayer film, thematerial thereof, and the order in which the layers are stacked differsin different cells. Thus, different spectral characteristics can beachieved in different cells. By using a multilayer film, sharp rise andfall of the spectral transmittance can be achieved. A configuration inwhich an organic material is used can be implemented by varying thepigment or dyestuffs to be contained in different cells or by stackinglayers of different kinds of materials. A configuration in which thediffraction grating structure is used can be implemented by providing adiffraction structure in which the diffraction pitch or depth differs indifferent cells. In a case in which a microstructure containing metal isused, the microstructure can be fabricated by utilizing dispersioncaused by the plasmon effect.

Second Embodiment

FIG. 4 is a schematic diagram illustrating an imaging apparatus D1according to a second embodiment. The imaging apparatus D1 according tothe present embodiment includes an imaging optical system 100, a codingelement C, and an image sensor S. The coding element C is identical tothe coding element C described in the first embodiment. Thus, detaileddescriptions of content that is similar to the content of the firstembodiment will be omitted.

FIG. 4 also depicts a signal processing circuit Pr that processes animage signal output from the image sensor S. The signal processingcircuit Pr may be incorporated into the imaging apparatus D1 or may be aconstituent element of a signal processing device that is electricallyconnected to the imaging apparatus D1 with a cable or wirelessly. Thesignal processing circuit Pr estimates a plurality of images F(hereinafter, also referred to as spectrally separated images ormulti-wavelength images) that are separated in the respective wavelengthbands of light from an object O, on the basis of an image G acquired bythe image sensor S.

The imaging optical system 100 includes at least one imaging lens.Although FIG. 4 depicts the imaging optical system 100 as a single lens,the imaging optical system 100 may be constituted by a combination of aplurality of lenses. The imaging optical system 100 forms an image on animaging surface of the image sensor S.

The coding element C is disposed in the vicinity of or immediately abovethe image sensor S. Here, being disposed in the vicinity means that thecoding element C is disposed in the proximity of the image sensor S suchthat a sufficiently sharp image of the light from the imaging opticalsystem 100 is formed on the surface of the coding element C. Beingdisposed immediately above means that the coding element C and the imagesensor S are dose to each other with little gap present therebetween.The coding element C and the image sensor S may be integrated into aunit. The coding dement C is a mask having a spatial distribution of anoptical transmittance. The coding element C transmits light that haspassed through the imaging optical system 100 and is incident on thecoding element C while modulating the intensity of that light.

FIG. 5A illustrates an example of the transmittance distribution of thecoding element C according to the present embodiment. This examplecorresponds to the binary-scale transmittance distribution describedabove. In FIG. 5A, the black portion indicates a region that transmitsalmost no light (referred to as a light-blocking region), and the whiteportion indicates a region that transmits light (referred to as alight-transmitting region). In this example, the optical transmittanceof the white portion is substantially 100%, and the opticaltransmittance of the black portion is substantially 0%. The codingelement C is divided into a plurality of rectangular regions, and eachrectangular region is either a light-transmitting region or alight-blocking region. The two-dimensional distribution of thelight-transmitting regions and the light-blocking regions in the codingelement C can, for example, be a random distribution or a quasi-randomdistribution.

The random distribution and the quasi-random distribution are consideredin a manner similar to the one described above. First, each region inthe coding element C can be considered as a vector element having avalue of, for example, 0 to 1 in accordance with the opticaltransmittance. In other words, a set of regions arrayed in a column canbe considered as a multi-dimensional vector having a value of 0 to 1.Therefore, it can be said that the coding element C includes a pluralityof multi-dimensional vectors arranged in the row direction. In thiscase, the random distribution means that any two multi-dimensionalvectors are independent from each other (they are not parallel).Meanwhile, the quasi-random distribution means that a configuration inwhich multi-dimensional vectors are not independent is included in someof the multi-dimensional vectors.

It can be said that the coding process by the coding element C is aprocess of performing marking for separating images of light atrespective wavelengths. As long as such marking is possible, thedistribution of the transmittance may be set as desired. In the exampleillustrated in FIG. 5A, the ratio of the number of the black portions tothe number of the white portions is 1 to 1, but an embodiment is notlimited to this ratio. For example, the distribution may be skewed suchthat the ratio of the number of the white portions to the number of theblack portions is 1 to 9.

FIG. 5B illustrates another configuration example of the coding elementC. This example corresponds to a mask having the gray-scaletransmittance distribution described above. In this case, each region inthe coding element C has a transmittance value at three or more levelsin accordance with the wavelength bands, as in the configurationillustrated in FIG. 1B.

As illustrated in FIGS. 5A and 5B, the coding element C has differenttransmittance spatial distributions in different wavelength bands W1,W2, . . . , and Wi. However, the transmittance spatial distributions inthe respective wavelength bands may coincide with one another whentranslated in the spatial direction.

FIG. 6A illustrates, in approximation, the spectral transmittance in agiven region of the coding element C that has the binary-scaletransmittance distribution illustrated in FIG. 5A. The region having thebinary-scale transmittance ideally has a spectral transmittance asillustrated in FIG. 6A. In this example, the spectral transmittance haslocal maxima in three wavelength bands among the plurality of wavelengthbands (each wavelength band is expressed by the interval between thegradations) included in the target wavelength band. In reality, thespectral transmittance is not like the ideal spectral transmittance asillustrated in FIG. 6A, but results in a continuous spectraltransmittance as illustrated in FIG. 6B. Even with the spectraltransmittance as illustrated in FIG. 6B, by approximating thetransmittance in a given wavelength band to substantially 1 when thevalue obtained by integrating the transmittance in each wavelength bandexceeds a predetermined threshold value, the spectral transmittance canbe regarded as the spectral transmittance illustrated in FIG. 6A.

Such information pertaining to the transmittance distribution of thecoding element C is acquired in advance as design data or throughmeasured calibration and is used in an operation process, which will bedescribed later.

The image sensor S is a monochrome image sensor having a plurality oflight-sensor cells (also referred to as pixels in the presentspecification) that are arrayed two-dimensionally. The image sensor Scan, for example, be a charge-coupled device (CCD) sensor, acomplementary metal-oxide semiconductor (CMOS) sensor, an infrared arraysensor, a terahertz array sensor, or a millimeter-wave array sensor. Alight-sensor cell can, for example, be constituted by a photodiode. Theimage sensor S does not need to be a monochrome image sensor. Forexample, a color image sensor having a filter of R/G/B, R/G/B/IR, orR/G/B/W may also be used. With the use of a color image sensor, theamount of information pertaining to the wavelengths can be increased,and the accuracy of reconstructing the spectrally separated images F canbe increased. However, if a color image sensor is used, the amount ofinformation in the spatial direction (x-, y-directions) is reduced, andthus there is a trade-off between the amount of information pertainingto the wavelengths and the resolution. The range of wavelengths to beacquired (target wavelength band) may be determined as desired, and thetarget wavelength band is not limited to a wavelength range of visiblelight and may be a wavelength range of ultraviolet, near-infrared,mid-infrared, far-infrared, microwaves, or radio waves.

The signal processing circuit Pr processes an image signal output fromthe image sensor S. The signal processing circuit Pr can, for example,be implemented by a combination of a computer program with a digitalsignal processor (DSP), a programmable logic device (PLD) such as afield programmable gate array (FPGA), or a central processing unit (CPU)and a graphics processing unit (GPU). Such a computer program is storedin a recording medium such as a memory, and as the CPU executes theprogram, the operation process described later can be executed. Asdescribed above, the signal processing circuit Pr may be a componentexternal to the imaging apparatus D1. In such a configuration, apersonal computer (PC) electrically connected to the imaging apparatusD1 or a signal processing device, such as a cloud server on theInternet, includes the signal processing circuit Pr. In the presentspecification, a system that includes such a signal processing deviceand the imaging apparatus is referred to as a spectroscopic system.

Hereinafter, the operation of the imaging apparatus D1 according to thepresent embodiment will be described.

FIG. 7 is a flowchart illustrating an overview of a spectroscopic methodaccording to the present embodiment. In step S101, the intensity ofincident light is spatially modulated in each wavelength band by usingthe coding element C. In step S102, an image in which components oflight that has passed through the coding element C are superimposed isacquired by the image sensor S. In step S103, a plurality of images inthe respective wavelength bands are generated on the basis of the imageacquired by the image sensor S and the spatial distribution of thespectral transmittance of the coding element C.

Subsequently, the process of acquiring a captured image G by the imagingapparatus D1 according to the present embodiment will be described.

Light rays from the object O are converged by the imaging optical system100, and the image of that light rays is coded by the coding element Cdisposed Immediately preceding the image sensor S. In other words, theintensity of light that passes through the coding element C is modulatedin accordance with the spatial distribution of the transmittance at therespective wavelengths in the coding element C. Consequently, imageshaving the coded information are formed on the imaging surface of theimage sensor S as a multiplex image in which the stated images aresuperimposed on one another. In this case, unlike the configuration ofthe conventional compressed sensing, a dispersive element such as aprism is not used, and thus the image does not shift in the spatialdirection. Therefore, a high spatial resolution can be retained even ina multiplex image. A plurality of black dots included in the image Gillustrated in FIG. 4 schematically represents low-luminance portionsgenerated through coding. The number and the disposition of the blackdots illustrated in FIG. 4 do not reflect the actual number anddisposition. In reality, the low-luminance portions can be generated ina greater number than those illustrated in FIG. 4. The information onthe multiplex image is converted to a plurality of electric signals(pixel signals) by the plurality of light-sensor cells in the imagesensor S, and the captured image G is thus generated.

The imaging apparatus D1 may further include a bandpass filter thattransmits only a component in some wavelength bands of incident lightrays. This makes it possible to limit a wavelength band to be measured.By limiting the wavelength band to be measured, spectrally separatedimages F with high separation precision within limited desiredwavelengths can be obtained.

Subsequently, a method for reconstructing multi-wavelength spectrallyseparated images F on the basis of the captured image G and the spatialdistribution characteristics of the transmittance at each wavelength ofthe coding element C will be described. Here, multi-wavelength means,for example, wavelength bands in a number greater than the number ofthree-color (R/G/B) wavelength bands acquired by a typical color camera.The number of the wavelength bands (hereinafter, also referred to as aspectral band number) is, for example, 4 to approximately 100. Dependingon the intended use, the spectral band number may exceed 100.

The data to be obtained is the spectrally separated images F, and thedata of the spectrally separated images F is represented by f. When thespectral band number (band number) is represented by w, f is the data inwhich pieces of image data f1, f2, . . . , and fw of the respectivebands are integrated. When the number of pixels in the x-direction ofthe image data to be obtained is represented by n and the number ofpixels in the y-direction is represented by m, each of the pieces of theimage data f1, f2, . . . , and fw is a set of two-dimensional datahaving n×m pixels. Therefore, the data f is three-dimensional datahaving n×m×w elements. Meanwhile, the number of elements in data g ofthe captured image G that has been coded and multiplexed by the codingelement C and is then acquired is n×m. The data g according to thepresent embodiment can be expressed by the following expression (2).

$\begin{matrix}{g = {{Hf} = {H\begin{bmatrix}f_{1} \\f_{2} \\\vdots \\f_{w}\end{bmatrix}}}} & (2)\end{matrix}$

Here, f1, f2, . . . , and fw are each data having n×m elements, and thusthe vector on the right-hand side is a one-dimensional vector of n×m×wrows by one column in a strict sense. The vector g is transformed andexpressed as a one-dimensional vector of n×m rows by one column and iscalculated. The matrix H expresses a transformation for coding theelements f1, f2, . . . , and fw of the vector f with the codinginformation that differs in the respective wavelength bands, modulatingthe intensity, and then adding the results. Therefore, H is a matrix ofn×m rows by n×m×w columns.

It seems that f can be calculated by solving an inverse problem of theexpression (2) when the vector g and the matrix H are given. However,since the number n×m×w of elements of the data f to be obtained isgreater than the number n×m of elements of the acquired data g, thisproblem is an ill-posed problem and cannot be solved as-is. Therefore,the signal processing circuit Pr according to the present embodimentfinds a solution through a compressed sensing technique by utilizing theredundancy of the image included in the data f. Specifically, the data fto be obtained is estimated by solving the following expression (3).

$\begin{matrix}{f^{\prime} = {\underset{f}{\arg\;\min}\left\{ {{{g - {Hf}}}_{l_{2}} + {{\tau\Phi}(f)}} \right\}}} & (3)\end{matrix}$

Here, f′ represents the data of estimated f. The first term inside thecurly braces in the above expression represents the amount of deviationbetween the estimation result Hf and the acquired data g, or in otherwords, is a residual term. Here, although the residual term is a sum ofsquares, the residual term may be an absolute value, a square root ofsum of squares, or the like. The second term inside the curly braces isa regularization term (or a stabilization term), which will be describedlater. The expression (3) is intended to obtain f that minimizes the sumof the first term and the second term. The signal processing circuit Prconverges the solution through a recursive iterative operation and cancalculate the final solution f′.

The first term inside the curly braces in the expression (3) correspondsto an operation of finding the sum of squares of a difference betweenthe acquired data g and Hf obtained by subjecting f in the estimationprocess to a system transformation by the matrix H. The expression φ(f)in the second term is a constraint in the regularization of f and is afunction that reflects sparse information of the estimated data. Theexpression acts to smooth or stabilize the estimated data. Theregularization term can, for example, be represented by the discretecosine transform (DCT) of f, the wavelet transform, the Fouriertransform, the total variation (TV), or the like. For example, if thetotal variation is used, stable estimated data in which an influence ofnoise of the observation data g is suppressed can be acquired. Thesparseness of the object O in the space of each regularization termdiffers depending on the texture of the object O. A regularization termin which the texture of the object O becomes sparser in the space of theregularization term may be selected. Alternatively, a plurality ofregularization terms may be included in an operation. The letter τ is aweighting factor, and as the value of τ is greater, the amount ofredundant data to be reduced becomes greater (compression rateincreases); and as the value of τ is smaller, the convergence toward thesolution is reduced. The weighting factor τ is set to an appropriatevalue such that f converges to a certain degree and is notovercompressed.

Here, although an operation example in which the compressed sensingindicated in the expression (3) is illustrated, another technique may beemployed to find a solution. For example, another statistical method,such as a maximum likelihood estimation method and a Bayes estimationmethod, can also be used. In addition, the number of the spectrallyseparated images F may be set to any number, and each wavelength bandmay also be set as desired.

As described thus far, in the present embodiment, the coding element Cwhose spectral transmittance characteristics differ in differentwavelength bands as illustrated in FIGS. 5A and 5B is used. Thus, asindicated in a first implementation example described later, anoccurrence of coma aberration can be suppressed, and a decrease in theresolution due to compressed sensing can be suppressed. According to thepresent embodiment, the three demands: a high resolution, multiplewavelengths, and moving image capturing (one-shot shooting) can befulfilled simultaneously. In addition, only two-dimensional data needsto be retained at imaging, and thus the present embodiment is effectivein acquiring data of an extended period of time. Although the imagesensor and the signal processing circuit according to the presentembodiment are configured to acquire a moving image, the image sensorand the signal processing circuit may be configured to acquire only astill image.

Third Embodiment

A third embodiment differs from the second embodiment in thatmulti-wavelength images are reconstructed by utilizing a blurred stateof a coding pattern on an image plane. Hereinafter, detaileddescriptions of content that is similar to the content of the secondembodiment will be omitted.

FIG. 8A is a schematic diagram illustrating an imaging apparatus D2according to the present embodiment. Unlike the imaging apparatus D1, inthe imaging apparatus D2, a coding element C is disposed so as to bespaced apart from an image sensor S. An image coded by the codingelement C is acquired in a blurred state on the image sensor S. Thus,this blur information is stored in advance and is reflected on thesystem matrix H of the expression (2). Here, the blur information isexpressed by a point spread function (PSF). The PSF is a function fordefining the degree of spread of a point image to surrounding pixels.For example, in a case in which a point image corresponding to one pixelon an image spreads to a region of k×k pixels surrounding theaforementioned pixel due to blurring, the PSF can be defined as acoefficient group (matrix) indicating an influence on the luminance ofeach pixel within the stated region. By reflecting the influence ofblurring of the coding pattern by the PSF onto the system matrix H,spectrally separated images F can be reconstructed.

The coding element C may be disposed at a desired position, but it isnecessary to prevent the coding pattern of the coding element C fromspreading too much and disappearing. Therefore, for example, asillustrated in FIG. 8B, the coding element C is preferably disposed inthe vicinity of a lens in an imaging optical system 100 that is closestto an object O or in the vicinity of the object O. In particular, anoptical system having a wide angle of view has a short focal length.Thus, light rays of the respective angles of view are less likelyoverlap one another at the aforementioned positions, and the codingpattern is less likely to be blurred on the image sensor S and is morelikely to be retained. In addition, if the coding element C is disposedat a position closer to the image sensor S as in the second embodiment,the coding pattern is more likely to be retained, and this ispreferable.

FIRST IMPLEMENTATION EXAMPLE

Subsequently, an implementation example of the present disclosure willbe described.

FIG. 9 illustrates an example of a result of reconstructing spectrallyseparated images F by using the spectroscopic method according to thepresent disclosure. Here, as the coding element C, an element having abinary pattern in which a plurality of light-transmitting regions and aplurality of light-blocking regions are arrayed randomly as illustratedin FIG. 5A is used. The optical transmittance of each of thelight-transmitting regions is substantially 100%, and the opticaltransmittance of each of the light-blocking regions is substantially 0%.The configuration of the imaging apparatus according to the secondembodiment is employed.

A captured image G is an image having 500×311 pixels in which 20 imagescoded in the respective wavelength bands by the coding element C aremultiplexed.

In the first implementation example, the spectrally separated images Fof the 20 wavelength bands are obtained by solving the estimationalgorithm of the expression (3) on the basis of the captured image G andthe spatial distribution characteristics of the transmittance in therespective wavelength bands of the coding element C. Here, the totalvariation (TV) is used as a regularization term.

FIRST COMPARATIVE EXAMPLE

As a comparative example, spectrally separated images F to be obtainedwhen, in place of the coding element C, a coding element whosetransmittance has almost no wavelength dependence and a dispersiveelement P that spectrally shifts per pixel in the y-direction are usedare reconstructed. The dispersive element is disposed in a path of lightthat has passed through the coding element and disperses the light into20 bands only in the y-direction.

FIG. 10 illustrates the result according to the comparative example. Acaptured image G illustrated in FIG. 10 is a multiplex image of theimages in the respective spectral bands shifted in the y-direction, andthus has a lower resolution than does the captured image G illustratedin FIG. 9. Consequently, the resolution of the spectrally separatedimages F is also low.

FIG. 11 illustrates a mean squared error (MSE) relative to a correctimage in each of the first implementation example and the firstcomparative example. The MSE is expressed by the expression (4) andindicates the mean square error per pixel. As the value is smaller, thespectrally separated images are closer to the correct image.

$\begin{matrix}{{MSE} = {\frac{1}{n \cdot m}{\sum\limits_{i = 1}^{n}{\sum\limits_{j = 1}^{m}\left( {I_{i,j}^{\prime} - I_{i,j}} \right)^{2}}}}} & (4)\end{matrix}$

Here, n and m represent the numbers of pixels, respectively, in thelongitudinal direction and in the lateral direction of the image,I′_(i,j) represents the value of the pixel in the ith row and the jthcolumn of a reconstructed image (spectrally separated image), andI_(i,j) represents the value of the pixel in the ith row and the jthcolumn of the correct image. The images used in the presentimplementation example and the comparative example are each an 8-bitimage, and the maximum value of the pixel value is 255.

The horizontal axis in FIG. 11 represents the image number of thereconstructed spectrally separated images F, and the vertical axisrepresents the value of the MSE. As can be seen from FIG. 11, in a casein which the images are reconstructed through the method of the firstimplementation example, as compared to the first comparative example,all of the spectrally separated images F have a small MSE value and areclose to the correct image. The values of the MSE are in a range fromapproximately 180 to 200, and it can be understood that the imagessubstantially match the correct image. In contrast, with the firstcomparative example, which is a conventional method, the values of theMSE are generally high, and it can be understood that the images arenotably deteriorated as compared to the correct image. The differencebetween the MSE in the first implementation example and the MSE in thefirst comparative example is 1.7 times at a minimum and 2.5 times at amaximum. Thus, the effectiveness of the configuration of the presentdisclosure can be recognized. This is because, unlike the firstcomparative example, in the first implementation example, an image shiftdoes not occur in the captured image G and the resolution of thecaptured image G is high. In addition, the configuration of the presentdisclosure does not require a relay optical system, and thus the sizecan be reduced to a great extent. Furthermore, with a spectral shiftmethod in which a prism or the like is used, saturation may occur in abroad range on a line when a high-intensity object such as a lightsource is imaged. In contrast, the range of saturation is limited in theembodiments of the present disclosure in which an image shift is notcarried out, which is thus advantageous.

Fourth Embodiment

A fourth embodiment of the present disclosure will be described withreference to FIGS. 12, 13A, 13B, 13C, 13D, 13E, 14A, and 14B. FIG. 12illustrates a spectroscopic system S1 according to the fourthembodiment. The definition of the spectroscopic system in the presentspecification will be given later. As illustrated in FIG. 12, thespectroscopic system S1 according to the present embodiment includes animaging apparatus D4 and a signal processing circuit Pr.

Imaging Apparatus D4

The imaging apparatus D4 according to the present embodiment includes anarrow-band coding device 200, an imaging lens 102, a spatiallymodulating coding element CS, and an image sensor S. The spatiallymodulating coding element CS corresponds to the first coding elementaccording to the present disclosure.

Narrow-Band Coding Device 200 The narrow-band coding device 200 isdisposed in an optical path of light rays R incident from an object O.Although the narrow-band coding device 200 is disposed between theobject O and the imaging lens 102 in the present embodiment, thenarrow-band coding device 200 may be disposed between the imaging lens102 and the spatially modulating coding element CS, which will bedescribed later. The narrow-band coding device 200 includes at least onenarrow-band coding element. The narrow-band coding element correspondsto the second coding element according to the present disclosure.Mode in Which Narrow-Band Coding Device 200 Includes Two Narrow-BandCoding Elements

The narrow-band coding device 200 includes a narrow-band coding elementC1 and a narrow-band coding element C2. The narrow-band coding device200 further includes a mechanism for switching between the narrow-bandcoding element C1 and the narrow-band coding element C2 for eachinstance of imaging. In the example illustrated in FIG. 12, thenarrow-band coding device 200 includes a mechanism for holding the twonarrow-band coding elements side by side in the direction perpendicularto the optical axis of the imaging apparatus D4. The narrow-band codingdevice 200 further includes a slide mechanism for moving the twonarrow-band coding elements in the direction perpendicular to theoptical axis of the imaging apparatus D4. The mechanism for switchingbetween the two narrow-band coding elements is not limited to the aboveexample. For example, the narrow-band coding device 200 may include arotation shaft. In this case, the two narrow-band coding elements aredisposed at an equal distance from the rotation shaft. As thenarrow-band coding device 200 rotates, a narrow-band coding element tobe disposed in the optical path is switched.

The narrow-band coding element C1 and the narrow-band coding element C2each have an optical transmittance that is uniform in the spatialdirection. Here, the uniform optical transmittance means that theoptical transmittance (or the wavelength distribution of the opticaltransmittance) is uniform or that an error in the optical transmittanceis within 10%.

In addition, the narrow-band coding element C1 and the narrow-bandcoding element C2 each include a plurality of light-transmitting regionsand a plurality of light-blocking regions in the wavelength direction inthe wavelength distribution of the optical transmittance. Hereinafter,this will be described in detail with reference to FIGS. 13A and 13B.

FIGS. 13A and 13B illustrate an example of the spectral transmittance ofthe narrow-band coding element C1 according to the fourth embodiment.Here, the spectral transmittance corresponds to the wavelengthdistribution of the optical transmittance. The spectral transmittance isexpressed by a function T(λ), in which λ represents the wavelength ofincident light. The spectral transmittance T(λ) can take a value no lessthan 0 and no greater than 1. In the example illustrated in FIGS. 13Aand 13B, the spectral transmittance of the narrow-band coding element C1changes periodically in the wavelength direction. One period of thespectral transmittance includes one each of a transmission wavelengthrange T in which the optical transmittance is substantially constant atsubstantially 1 and a light-blocking wavelength range Q in which theoptical transmittance is substantially constant at substantially 0. Inthe present specification, the transmission wavelength range T isdefined as a wavelength range in which the optical transmittance is noless than 0.5 and the mean transmittance therewithin is substantially 1.In a similar manner, the light-blocking wavelength range Q is defined asa wavelength range in which the optical transmittance is less than 0.5and the mean transmittance therewithin is substantially 0. Substantially1 corresponds to a value that is no less than 0.8, and substantially 0corresponds to a value no greater than 0.2. The spectral transmittanceof the narrow-band coding element desirably has a binary distribution.The binary distribution is a distribution in which the transmittance inthe transmission wavelength range T is constant at substantially 1 andthe transmittance in the light-blocking wavelength range Q is constantat substantially 0, However, the spectral transmittance does not have tohave a binary distribution as long as the mean of the transmittance inthe transmission wavelength range T is substantially 1 and the mean ofthe transmittance in the light-blocking wavelength range Q issubstantially 0. The same applies to the narrow-band coding elementdescribed hereinafter.

In the example of the spectral transmittance illustrated in FIGS. 13Aand 13B, the band widths of the transmission wavelength range T and thelight-blocking wavelength range Q are set to be equal to each other, butan embodiment is not limited to this example. As illustrated in FIGS.13C to 13E, the band width of the transmission wavelength range T may beless than the band width of the light-blocking wavelength range Q. Inaddition, the spectral transmittance does not have to changeperiodically. In other words, the plurality of transmission wavelengthranges T in the spectral transmittance may have mutually different bandwidths. In addition, the plurality of light-blocking wavelength ranges Qin the spectral transmittance may have mutually different band widths.

FIG. 14A illustrates a relation between a target wavelength band W and aplurality of wavelength bands W1, W2, . . . , and Wn included in thetarget wavelength band W. The target wavelength band W is a wavelengthband in which the imaging apparatus D4 is to capture an image. Theplurality of wavelength bands W1, W2, . . . , and Wn are wavelengthbands to be used by the imaging apparatus D4 for image signals. Inaddition, the signal processing circuit Pr, which will be describedlater, reconstructs n spectrally separated images F1, F2, . . . , and Fnthat are separated in the respective wavelength bands W1 W2, . . . , andWn.

The target wavelength band W can be set in a variety of ranges inaccordance with the intended use. The target wavelength band W can, forexample, be a visible-light wavelength band (approximately 400 nm toapproximately 700 nm), a near-infrared wavelength band (approximately700 nm to approximately 2500 nm), a near-ultraviolet wavelength band(approximately 10 nm to approximately 400 nm), a mid-infrared band, afar-infrared band, or a radio wave range including terahertz waves andmillimeter waves. In this manner, the wavelength band to be used in theimaging apparatus D4 is not limited to a visible-light band. In thepresent specification, aside from visible light, non-visible raysincluding near-ultraviolet rays, near-infrared rays, and radio waves arealso referred to as light for convenience.

In the present embodiment, as illustrated in FIG. 14A, with n being anyinteger of no less than 4, wavelength bands obtained by equally dividingthe target wavelength band W by n are designated as the wavelength bandsW1, W2, . . . , and Wn. However, the setting of the plurality ofwavelength bands to be included in the target wavelength band W is notlimited to the above example. For example, the widths of the wavelengthbands (band widths) may vary. As illustrated in FIG. 14B, the bandwidths may vary among the wavelength bands, or a gap may be presentbetween two adjacent wavelength bands. In this manner, the plurality ofwavelength bands may be set as desired as long as there is no portion inwhich wavelength bands overlap each other. The number n of thewavelength bands is, for example, 4 to approximately 100, but the numbern may exceed 100 depending on the intended use.

In the present embodiment, the width of the transmission wavelengthranges T in the narrow-band coding element C1 and the narrow-band codingelement C2 is designed to substantially match the width of thewavelength bands W1, W2, . . . , and Wn. The width of each wavelengthband is, for example, 20 nm. The width of each wavelength band may be 10nm, 5 nm, or 1 nm.

Thus, the narrow-band coding element C1 includes a plurality oflight-transmitting regions (transmission wavelength ranges T) and aplurality of light-blocking regions (light-blocking wavelength ranges Q)in the wavelength direction in the wavelength distribution of theoptical transmittance. The plurality of light-transmitting regions eachcorrespond to one of the wavelength bands W1 W2, . . . , and Wn.

FIG. 13B illustrates the spectral transmittance of the narrow-bandcoding element C2. The spectral transmittance illustrated in FIG. 13Bhas a distribution in which the distribution of the transmissionwavelength ranges T and the light-blocking wavelength ranges Q in thespectral transmittance of the narrow-band coding element C1 illustratedin FIG. 13A is inverted. Therefore, the wavelength range that includesthe transmission wavelength ranges T in the spectral transmittance ofthe narrow-band coding element C1 and the transmission wavelength rangesT in the spectral transmittance of the narrow-band coding element C2covers the entirety of the plurality of wavelength bands W1, W2, . . . ,and Wn.

Thus, the narrow-band coding element C1 and the narrow-band codingelement C2 have the respective pluralities of light-transmitting regions(transmission wavelength ranges T) in mutually different wavelengthbands. In addition, the plurality of light-transmitting regions eachcorrespond to one of the wavelength bands W1, W2, . . . , and Wn.

Mode in Which Narrow-Band Coding Device 200 Includes Three Narrow-BandCoding Elements

The narrow-band coding device 200 may include three narrow-band codingelements, namely, a narrow-band coding element C11, a narrow-band codingelement C12, and a narrow-band coding element C13. Examples of thespectral transmittances of the respective narrow-band coding elementsC11, C12, and C13 are illustrated in FIGS. 13C, 13D, and 13E. The threespectral transmittances illustrated in FIGS. 13C, 13D, and 13E each havea periodical distribution in the wavelength direction and a uniformdistribution in the spatial direction. In addition, in each of thespectral transmittances, one period includes one each of thetransmission wavelength range T and the light-blocking wavelength rangeQ.

The spectral transmittance of the narrow-band coding element C12illustrated in FIG. 13D is obtained by shifting the spectraltransmittance of the narrow-band coding element C11 illustrated in FIG.13C by the width of the transmission wavelength range T in the directionin which the wavelength increases. In a similar manner, the spectraltransmittance of the narrow-band coding element C13 illustrated in FIG.13E is obtained by shifting the spectral transmittance of thenarrow-band coding element C11 illustrated in FIG. 13C by twice thewidth of the transmission wavelength range T in the direction in whichthe wavelength increases. Therefore, the wavelength range that includesthe transmission wavelength ranges T in the spectral transmittances ofthe narrow-band coding element C11, the narrow-band coding element C12,and the narrow-band coding element C13 covers the entirety of theplurality of wavelength bands W1, W2, . . . , and Wn.

In the examples of the spectral transmittances illustrated in FIGS. 13C,13D, and 13E, the transmission wavelength range T has a width that ishalf the width of the light-blocking wavelength range Q, but anembodiment is not limited to this example. In a similar manner, thethree spectral transmittances have the transmission wavelength ranges Tof equal width, but an embodiment is not limited to this example. Inaddition, the three spectral transmittances have the light-blockingwavelength ranges T of equal width, but an embodiment is not limited tothis example. Although the spectral transmittance changes periodically,an embodiment is not limited to this example. In other words, theplurality of transmission wavelength ranges T in one spectraltransmittance may have mutually different wavelength band widths. Inaddition, the plurality of light-blocking wavelength ranges Q in onespectral transmittance may have mutually different wavelength bandwidths. Thus, it is sufficient that the spectral transmittances of thenarrow-band coding element C11, the narrow-band coding element C12, andthe narrow-band coding element C13 cover the target wavelength band Wwhen all of the transmission wavelength ranges T in the three spectraltransmittances are combined.

Other Modes of Narrow-Band Coding Device 200

The narrow-band coding device 200 may include four or more narrow-bandcoding elements. In that case, the narrow-band coding elements aredesigned such that the wavelength range including the transmissionwavelength ranges T in the spectral transmittances of the narrow-bandcoding element covers the entire wavelength bands W1, W2, . . . , Wn.

Alternatively, as illustrated in FIG. 15, the imaging apparatus D4 mayinclude only one narrow-band coding element CN, in place of thenarrow-band coding device 200. In this case, the imaging apparatus D4uses a wavelength band corresponding to the transmission wavelengthrange T in the spectral transmittance of the narrow-band coding elementCN for an image signal. In other words, the imaging apparatus D4 doesnot use a wavelength band corresponding to the light-blocking wavelengthrange Q in the spectral transmittance of the narrow-band coding elementCN for an image signal. Therefore, the entirety of the plurality ofwavelength bands W1, W2, . . . , and Wn is covered by the transmissionwavelength ranges T in the spectral transmittance of the singlenarrow-band coding element CN.

Structure of Narrow-Band Coding Element

The narrow-band coding element can be constituted by at least one of amultilayer film, an organic material, a diffraction grating structure,and a microstructure containing metal. In a case in which a multilayerfilm is used, for example, a dielectric multilayer film or a multilayerfilm that includes a metal layer can be used. By using the multilayerfilm, sharp rise and fall of the transmittance at a boundary between thetransmission wavelength range T and the light-blocking wavelength rangeQ can be achieved. The configuration in which an organic material isused can be implemented by stacking materials that contain differentpigments or dyestuffs. The configuration in which the diffractiongrating structure is used can be implemented by providing a diffractionstructure in which the diffraction pitch or depth is adjusted. In a casein which a microstructure containing metal is used, the microstructurecan be fabricated by utilizing dispersion caused by the plasmon effect.

Imaging Lens 102

The imaging lens 102 converges light from the object O and forms animage on the imaging surface of the image sensor S. In place of theimaging lens 102, an imaging optical system constituted by a combinationof a plurality of imaging lenses may be used.

Spatially Modulating Coding Element CS

The spatially modulating coding element CS is disposed in an opticalpath of light incident from the object O.

The spatially modulating coding element CS includes a plurality ofregions arrayed two-dimensionally. For example, as illustrated in FIG.16A, the spatially modulating coding element CS includes a plurality ofregions A divided in a lattice pattern. Each of the regions A is formedby a light-transmitting member. Although FIG. 16A illustrates 48rectangular regions A arrayed in six rows by eight columns, in an actualuse, a greater number of regions A can be provided. The number of theregions A corresponds to the number of pixels in a typical image sensorand can, for example, be approximately several hundred thousand toseveral ten million. In the present embodiment, the spatially modulatingcoding element CS is disposed immediately above the image sensor S. Inaddition, the interval among the plurality of regions A substantiallymatches the pixel pitch of the image sensor. Thus, light that has passedthrough a given region A is incident only on one corresponding pixel ofthe image sensor S. In addition, the spatially modulating coding elementCS may be disposed so as to be spaced apart from the image sensor S. Inthis case, the interval of the regions A may be reduced in accordancewith the distance from the image sensor S. Thus, light that has passedthrough a given region A of the spatially modulating coding element CScan be made to be incident only on one corresponding pixel of the imagesensor S.

FIG. 16B illustrates an example of the spatial distribution of theoptical transmittance in the spatially modulating coding element CS. Theoptical transmittance is indicated for each of the plurality ofwavelength bands W1, W2, . . . , and Wn. In FIG. 16B, the difference inthe shade of gray among the regions A represents the difference in theoptical transmittance. A lighter gray region has a higher opticaltransmittance, and a darker gray region has a lower opticaltransmittance. The spatially modulating coding element CS includes, ineach of the plurality of wavelength bands W1, W2, . . . , and Wn, aplurality of regions A having an optical transmittance of substantially1 and a plurality of regions A having an optical transmittance ofsubstantially 0. The spatially modulating coding element CS may furtherinclude a plurality of regions A having a third optical transmittancethat lies between substantially 0 and substantially 1. The spatiallymodulating coding element CS is formed by a plurality oflight-transmitting regions and a plurality of light-blocking regionsthat are disposed spatially in each of the plurality of wavelength bandsW1, W2, . . . , and Wn. A light-transmitting region is a region A thathas an optical transmittance of no less than 0.5 and, for example, hasan optical transmittance of substantially 1 or the third opticaltransmittance. A light-blocking region is a region A that has an opticaltransmittance of less than 0.5 and, for example, has an opticaltransmittance of substantially 0. As illustrated in FIG. 16B, thespatial distribution of the plurality of light-transmitting regions andthe plurality of light-blocking regions differ among differentwavelength bands W1, W2, . . . , and Wn.

Thus, the spatially modulating coding element CS includes a plurality oflight-transmitting regions (regions A) and a plurality of light-blockingregions (regions A) in the spatial direction in the spatial distributionof the optical transmittance. In addition, the spatial distribution ofthe plurality of light-transmitting regions and the plurality oflight-blocking regions differ among different wavelength bands W1, W2, .. . , and Wn.

FIGS. 16C and 16D illustrate the spectral transmittances of,respectively, a region A1 and a region A2, which are two representativeregions A among the plurality of regions A included in the spatiallymodulating coding element CS according to the present embodiment.

The spectral transmittance in each of the regions A of the spatiallymodulating coding element CS has a lower resolving power in thewavelength direction than does the spectral transmittance of thenarrow-band coding element illustrated in FIGS. 13A and 13B.Hereinafter, the resolving power of the spectral transmittance in theregions A will be described in more concrete terms with the region A1serving as an example.

FIG. 17 illustrates the spectral transmittance in the region A1 of thespatially modulating coding element CS. The spectral transmittanceillustrated in FIG. 17 is normalized such that the optical transmittancewithin the target wavelength band W has a maximum value of 1 and aminimum value of 0. The spectral transmittance illustrated in FIG. 17has local maxima M1, M2, M3, M4, and M5 and local minima m1, m2, m3, m4,m5, and m6. As can be seen from FIG. 17, the local maxima M1, M3, M4,and M5 are no less than 0.5. For example, the local maximum M1 isadjacent to the local minimum m1 and the local minimum m2. At awavelength λ1, the transmittance takes a mean value of the local minimumm1 and the local maximum M1. In addition, at a wavelength λ2, thetransmittance takes a mean value of the local minimum m2 and the localmaximum M1. In this case, a wavelength band width corresponding to thewavelength λ2-the wavelength λ1 corresponds to the resolving power inthe wavelength direction. In the spatially modulating coding element CS,at least part of the wavelength band width corresponding to theresolving power is greater than the width of the wavelength bands W1,W2, . . . , and Wn. Meanwhile, as described above, the narrow-bandcoding element includes the transmission wavelength range T having awidth equivalent to the width of the wavelength bands W1, W2, . . . ,and Wn. The transmission wavelength range T corresponds to the resolvingpower of the narrow-band coding element. Therefore, it can be said thatthe spectral transmittance in the region A1 of the spatially modulatingcoding element CS illustrated in FIG. 17 has a lower resolving power inthe wavelength direction than does the spectral transmittance of thenarrow-band coding element.

Subsequently, an example of a condition that the spectral transmittancein each region A is to satisfy will be described. Here, the example isdescribed using the characteristics of the spectral transmittance inwhich the spectral transmittance of the narrow-band coding elementdescribed above and the spectral transmittance of the region A of thespatially modulating coding element are superimposed on each other(hereinafter, may simply be referred to as the superimposed spectraltransmittance).

FIG. 18A illustrates the spectral transmittance of the narrow-bandcoding element, and FIG. 18B illustrates the spectral transmittance in agiven region A1 of the spatially modulating coding element CS. Inaddition, FIG. 18C illustrates a graph in which the spectraltransmittances illustrated in FIGS. 18A and 18B are superimposed on eachother, and portions in which the light-transmitting regions overlap areindicated by hatching. This hatched portions indicate the wavelength andthe intensity of light that has a uniform intensity distribution in thetarget wavelength band W and is transmitted upon being incident on thenarrow-band coding element and the region A1 of the spatially modulatingcoding element CS. Here, the mean value of the transmittances in therespective wavelength bands W1, W2, . . . , and Wn is obtained byintegrating the transmittances within the range of each of thewavelength bands and by dividing the result by the width of thewavelength band. In the present disclosure, the mean value of thetransmittances in each wavelength band obtained through this method isreferred to as a mean transmittance of that wavelength band.

FIG. 19 illustrates only the portions indicated by hatching in thespectral transmittance illustrated in FIG. 18C. In the presentembodiment, in the spectral transmittance illustrated in FIG. 19, themean transmittance is no less than 0.5 in at least two or morewavelength bands among the wavelength bands W1, W2, . . . , and Wn. Forexample, in FIG. 19, the mean transmittance is no less than 0.5 in thewavelength band W4, the wavelength band Wn-1, and so on.

When the spectral transmittance of another region A different from theregion A1 and the spectral transmittance of the narrow-band codingelement are superimposed on each other, a spectral transmittancedifferent from the one illustrated in FIG. 19 is obtained. Specifically,the combination or the number of wavelength bands in which the meantransmittance is no less than 0.5 differs.

Thus, with regard to the spectral transmittance in each region A, it issufficient that the superimposed spectral transmittance has a meantransmittance of no less than 0.5 in two or more wavelength bands andthe spectral transmittances in the regions A have mutually differentspectral transmittances.

In addition, the spectral transmittance in each region A may satisfy acondition 1 or a condition 2 described hereinafter.

In the ith row of the spatially modulating coding element CS (1≦i≦6), aset X of the regions A arrayed in one line in the row direction will beconsidered. In each region A in the set X, a vector Ai(j) of one row byeight columns having, as its elements, the values of the superimposedmean transmittances in a wavelength band Wj (1≦j≦n) in the regions A ofthe set X will be considered. The superimposed mean transmittancecorresponds to a mean transmittance in the spectral transmittance inwhich the spectral transmittance in each region A and the spectraltransmittance of the narrow-band coding element are superimposed on eachother. In the present embodiment, this vector Ai(j) is independent forany given j, or in other words, among any given wavelength bands(condition 1). For example, in a set X1 of the regions A in the firstrow, a vector having, as its elements, the values of the superimposedmean transmittances in the wavelength band W1 is expressed as A1(1). Ina similar manner, in the set X1 of the regions A in the first row, avector having, as its elements, the values of the superimposed meantransmittances in the wavelength band W2 is expressed as A1(2). Thevector A1(1) and the vector A1(2) are independent from each other.Vectors may be independent from each other in all the combinations ofthe wavelength bands W1, W2, . . . , and Wn. Alternatively, vectors maybe independent from each other among some of the wavelength bands W1,W2, . . . , and Wn.

In addition, in the present embodiment, the vector Ai(j) is independentfor any given i, or in other words, independent from any given row(condition 2). For example, in the wavelength band W1, a vector having,as its elements, the values of the superimposed mean transmittances in aset X1 of the regions A in the first row is expressed as A1(1). In asimilar manner, in the wavelength band W1, a vector having, as itselements, the values of the superimposed mean transmittance in a set X2of the regions A in the second row is expressed as A2(1) The vectorA1(1) and the vector A2(1) are independent from each other. Vectors maybe independent from each other in all the combinations of the rows.Vectors may be independent from each other in some combinations of therows.

A binary-scale spectral transmittance may be employed. The binary-scalespectral transmittance is defined such that the mean transmittance takesa value of either substantially 1 or substantially 0.

An example of the binary-scale spectral transmittance is illustrated inFIG. 20A. The spectral transmittance illustrated in FIG. 20A takes avalue of either 0 or 1 in the entire wavelength bands. Meanwhile, thespectral transmittance as illustrated in FIG. 20B is also a binary-scalespectral transmittance.

FIGS. 21A and 21B illustrate another configuration example of thespatially modulating coding element CS according to the presentembodiment. FIGS. 21A and 21B illustrate a configuration having agreater number of regions A than does the configuration of the exampleillustrated in FIG. 16A. FIG. 21A illustrates an example of thetransmittance distribution of the spatially modulating coding element CSin which each region A has a binary-scale spectral transmittance. InFIG. 21A, a portion of the spatially modulating coding element CS isenclosed by a rectangular and enlarged, and the transmittancedistribution in the enclosed portion is illustrated for each wavelengthband. In FIG. 21A, a black portion indicates a light-blocking region inwhich the optical transmittance is substantially 0. Meanwhile, a whiteportion indicates a light-transmitting region in which the opticaltransmittance is substantially 1. In the example illustrated in FIG.21A, the ratio of the number of the light-blocking regions to the numberof the light-transmitting regions is 1 to 1, but an embodiment is notlimited to this ratio. For example, the distribution may be skewed suchthat the ratio of the number of the light-transmitting regions to thenumber of the light-blocking regions is 1 to 9.

FIG. 21B illustrates an example of the spatially modulating codingelement CS in which each region A has a continuous spectraltransmittance.

As illustrated in FIGS. 21A and 21B, the spatially modulating codingelement CS has different transmittance spatial distributions in thedifferent wavelength bands W1, W2, . . . , and Wn. However, the spatialdistribution of the transmittance in a predetermined wavelength band ina portion including a plurality of regions may match the spatialdistribution of the transmittance in the stated wavelength band inanother portion.

Some of the entire regions A of the spatially modulating coding elementCS may be transparent regions. A transparent region in the presentspecification corresponds to a region that transmits light in the entirewavelength bands W1 to Wn included in the target wavelength band W at asubstantially equal high transmittance (e.g., 0.8 or more). For example,half of the entire regions A may be transparent regions and thetransparent regions may be disposed in a checkered pattern. In otherwords, in the two array directions (the lateral direction and thelongitudinal direction in FIG. 16A) of the plurality of regions A in thespatially modulating coding element CS, regions A whose spectraltransmittance differs depending on the wavelength band and thetransparent regions can be arrayed in an alternating manner.

The spatially modulating coding element CS can be constituted by atleast one of a multilayer film, an organic material, a diffractiongrating structure, and a microstructure containing metal. In a case inwhich a multilayer film is used, for example, a dielectric multilayerfilm or a multilayer film that includes a metal layer can be used. Inthis case, the cells are formed such that at least one of the thicknessof the multilayer film, the materials, and the order in which the layersare stacked differs among the different cells. Thus, different spectralcharacteristics can be achieved in different regions A. In a case inwhich each region A has a binary-scale spectral transmittance, by usingthe multilayer film, sharp rise and fall of the spectral transmittancecan be achieved. The configuration in which an organic material is usedcan be implemented by using different pigment or dyestuffs to becontained in different regions A or by stacking layers of differentkinds of materials. The configuration in which the diffraction gratingstructure is used can be implemented by providing a diffractionstructure in which the diffraction pitch or depth differs in differentregions A. In a case in which a microstructure containing metal is used,the microstructure can be fabricated by utilizing dispersion caused bythe plasmon effect.

Image Sensor S

The image sensor S is a monochrome image sensor having a plurality oflight-sensor cells (also referred to as pixels in the presentspecification) that are arrayed two-dimensionally. The image sensor Scan, for example, be a charge-coupled device (CCD) sensor, acomplementary metal-oxide semiconductor (CMOS) sensor, an infrared arraysensor, a terahertz array sensor, or a millimeter-wave array sensor. Alight-sensor cell can, for example, be constituted by a photodiode. Asthe image sensor S, for example, a color image sensor having a filter ofR/G/B, R/G/B/IR, or R/G/B/W may also be used. With the use of a colorimage sensor, the amount of information pertaining to the wavelengthscan be increased, and the accuracy of reconstructing the spectrallyseparated images F can be increased.

Signal Processing Circuit Pr

The signal processing circuit Pr estimates the spectrally separatedimages F on the basis of the captured image G acquired by the imagesensor S.

The signal processing circuit Pr processes an image signal output fromthe image sensor S. The signal processing circuit Pr can, for example,be implemented by a combination of a computer program with a digitalsignal processor (DSP), a programmable logic device (PLO) such as afield programmable gate array (FPGA), or a central processing unit (CPU)or a graphics processing unit (CPU). Such a computer program is storedin a recording medium such as a memory, and as the CPU executes theprogram, the operation process described later can be executed. Thesignal processing circuit Pr may be a constituent element of a signalprocessing device that is electrically connected to the imagingapparatus D4 with a cable or wirelessly. In such a configuration, apersonal computer (PC) electrically connected to the imaging apparatusD4 or a signal processing device, such as a cloud server on theInternet, includes the signal processing circuit Pr. In the presentspecification, a system that includes such a signal processing deviceand the imaging apparatus is referred to as a spectroscopic system. Thesignal processing circuit Pr acquires such information pertaining to thetransmittance distributions of the narrow-band coding element and thespatially modulating coding element CS in advance as design data orthrough measured calibration and uses the information in operations andprocessing, which will be described later.

Measured calibration will be described. For example, in a case in whichthe target wavelength band is the entire visible light range, a whiteboard serving as an object is disposed at the position of the object O,and white light from the object O is made to pass through thenarrow-band coding element or the spatially modulating coding elementCS. Thus, an image of the narrow-band coding element or the spatiallymodulating coding element CS can be formed on the image sensor S. It ispossible to calculate how each region A modulates white light thatequally contains wavelength components of the entire visible light fromthe image of the narrow-band coding element or the spatially modulatingcoding element CS, or in other words, it is possible to calculate thespectral transmittance of each region A. In addition, light whosewavelength band has been limited by a bandpass filter may be used.Through multiple instances of imaging with the plurality of bandpassfilters being replaced, transmittance data in the entire desiredwavelength bands may be acquired. Some of the wavelength bands may beselected and measured, and the transmittance data in the otherwavelength bands may be calculated through interpolation of the measureddata.

Other Configuration

The imaging apparatus D4 may further include a bandpass filter. Thebandpass filter transmits only the target wavelength band W of thereflected light from the object O. Thus, components of wavelength bandsother than the target wavelength band W that are not removed by thenarrow-band coding element or the spatially modulating coding elementcan be removed. Thus, the spectrally separated images F with highseparation precision only in a desired target wavelength band W can beobtained.

Operation

Hereinafter, the operation of the imaging apparatus D4 according to thepresent embodiment will be described with reference to FIG. 22. FIG. 22is a flowchart illustrating an overview of the spectroscopic method withthe use of the spectroscopic system S1 according to the presentembodiment.

In step 1X, the narrow-band coding element C1 is disposed in an opticalpath of the object O.

In step 1A, the intensity of incident light is spatially modulated ineach wavelength band by using both the narrow-band coding element C1 andthe spatially modulating coding element CS. This process is referred toas coding in the present specification. Specifically, light rays R fromthe object O are incident on the narrow-band coding element C1. Of thelight incident on the narrow-band coding element C1, only light having awavelength within the transmission wavelength range T passes through thenarrow-band coding element C1, and light having a wavelength within thelight-blocking wavelength range Q is blocked. Thus, the light rays R aremodulated to light having a plurality of intensity peaks that arediscrete relative to the wavelengths and is converged by the imaginglens 102. The light that has been converged by the imaging lens 102 isincident on the spatially modulating coding element CS. The light thathas been modulated by the narrow-band coding element C1 is incident oneach of the plurality of regions A of the spatially modulating codingelement CS. Each region A modulates the light having a plurality ofintensity peaks included in the incident light in accordance with thespectral transmittance of each region A and outputs the result. Asdescribed above, the narrow-band coding element C1 and the spatiallymodulating coding element CS have the spectral transmittancecharacteristics illustrated in FIG. 20A or 20B relative to the targetwavelength band W. Therefore, the narrow-band coding element C1 and thespatially modulating coding element CS superimpose light in at least twoof the wavelength bands W1, W2, . . . , and Wn to be separated andoutputs the result to the image sensor S. This means that datacompressed in the wavelength direction is acquired.

Subsequently, in step 1B, the captured image G is generated from thelight that has passed through the narrow-band coding element C1 and thespatially modulating coding element CS and is incident on the imagesensor S. Specifically, the light that is incident on the plurality ofpixels of the image sensor S is converted to a plurality of electricsignals (pixel signals). A set of the plurality of converted pixelsignals is the captured image G. An example of the captured image G isillustrated in FIG. 12. A plurality of black dots included in thecaptured image G illustrated in FIG. 12 schematically representlow-luminance portions generated through coding. The number and thedisposition of the black dots illustrated in FIG. 12 do not reflect theactual number and disposition. In reality, the low-luminance portionscan be generated in a greater number than those illustrated in FIG. 12.

Thereafter, at the branching Y, it is determined whether imaging hasbeen carried out by using all of the narrow-band coding elements. Ifimaging has not been carried out by using all of the narrow-band codingelements, the process proceeds to step 1D.

In step 1D, the narrow-band coding element C1 is replaced with thenarrow-band coding element C2. Thereafter, step 1X, step 1A, and step 1Bare carried out again using the replaced narrow-band coding element C2.

In a case in which the narrow-band coding device includes three or morenarrow-band coding elements, the cycle of step 1D, step 1X, step 1A, andstep 1B is repeated until imaging with all of the narrow-band codingelements is finished. In a case in which the narrow-band coding deviceincludes only one narrow-band coding element, step 1X, step 1A, and step1B are each carried out once.

When imaging with all of the narrow-band coding elements is completed,the process proceeds to step 1C.

In step 10, the signal processing circuit Pr generates the spectrallyseparated images F on the basis of the captured image G acquired by theimage sensor S, the wavelength distribution information of the opticaltransmittance of the narrow-band coding element, and the spatialdistribution information and the wavelength distribution information ofthe optical transmittance of the spatially modulating coding element CS.

The method for generating the spectrally separated images F in step 1Cwill be described in more concrete terms.

The data of the spectrally separated images F is indicated as aspectrally separated image f, and the data of the acquired capturedimage G is indicated as a captured image g. The captured image g can beexpressed by the following expression (5) that includes the spectrallyseparated image f.

$\begin{matrix}{g = {{{Hf}\begin{bmatrix}g_{1} \\g_{2} \\\vdots \\g_{m}\end{bmatrix}} = {\begin{bmatrix}{h_{1}\left( w_{1} \right)} & {h_{1}\left( w_{2} \right)} & \ldots & {h_{1}\left( w_{n} \right)} \\{h_{2}\left( w_{1} \right)} & {h_{2}\left( w_{2} \right)} & \; & {h_{2}\left( w_{n} \right)} \\\vdots & \; & \ddots & \vdots \\{h_{m}\left( w_{1} \right)} & {h_{m}\left( w_{2} \right)} & \ldots & {h_{m}\left( w_{n} \right)}\end{bmatrix}\begin{bmatrix}{f_{1}\left( w_{1} \right)} \\{f_{2}\left( w_{2} \right)} \\\vdots \\{f_{n}\left( w_{n} \right)}\end{bmatrix}}}} & (5)\end{matrix}$

In the expression (5), the spectrally separated image f is indicated asa vector having, as its elements, image data f1, f2, . . . , and fn ofthe respective wavelength bands W1, W2, . . . , and Wn. In addition, thecaptured image g is expressed as a vector having, as its elements, imagedata g1, g2, . . . , and gn acquired in the respective instances ofimaging. In the following description, the terms the spectrallyseparated image vector f and the captured image vector g may be used insome cases.

When the number of pixels in the x-direction of the spectrally separatedimages F to be obtained is represented by px and the number of pixels inthe y-direction is represented by py, each pieces of the image data f1,f2, . . . , and fn of the respective wavelength bands hastwo-dimensional data of px×py pixels. The spectrally separated image fhas three-dimensional data with px×py×n elements.

Meanwhile, when the imaging is carried out m times with the mnarrow-band coding elements being replaced one after another, thecaptured image g has three-dimensional data with px×py×m elements.

In the expression (4), the matrix H expresses a transformation in whichthe image data f1, f2, . . . , and fn of the respective wavelengthbands, which are elements of the spectrally separated image vector f,are coded with coding information that differs in different wavelengthbands and the obtained results are added. The matrix H is a matrix ofpx×py×m rows by px×py×n columns. Its matrix element hi(wj) (1≦i≦m,1≦j≦n) is expressed by the product of the optical transmittance in awavelength band wj of a narrow-band coding element Ci used for imagingin an imaging time Ti and the spatial distribution of the opticaltransmittance of the spatially modulating coding element CS. In thepresent embodiment, when the narrow-band coding device 200 includes twonarrow-band coding elements as illustrated in FIGS. 13A and 13B, forexample, the matrix element h1(wj) is 0 when j is odd, and the matrixelement h2(wj) is 0 when j is even. In a similar manner, when thenarrow-band coding device 200 includes three or more narrow-band codingelements, the matrix H regularly contains 0 components. When the matrixH contains a large number of 0 components, the number of unknowns to beconsidered in the operation process is reduced.

Here, the image data f1, f2, . . . , and fn in the respective wavelengthbands W1, W2, . . . , and Wn are data each having px×py elements, andthus the spectrally separated image f expressed as a vector in theright-hand side corresponds to a one-dimensional vector of px×py×n rowsby one column in a strict sense. In this case, the captured image g canbe converted to and expressed as a one-dimensional vector of px×py×mrows by one column.

It seems that the spectrally separated image vector f can be calculatedby solving an inverse problem of the expression (5) if the capturedimage vector g and the matrix H are given. However, m is less than n,and the number px×py×n of elements in the spectrally separated imagevector f to be obtained is greater than the number px×py×m of elementsin the acquired captured image vector g. Therefore, this problem is anill-posed problem and cannot be solved as-is. Thus, the signalprocessing circuit Pr according to the present embodiment finds asolution through the compressed sensing technique by utilizing theredundancy of the image included in the spectrally separated image f.Specifically, the spectrally separated image vector f to be obtained isestimated by solving the following expression (6).

$\begin{matrix}{f^{\prime} = {\underset{f}{\arg\;\min}\left\{ {{{g - {Hf}}}_{l_{2}} + {{\tau\Phi}(f)}} \right\}}} & (6)\end{matrix}$

Here, an estimated image vector f′ represents the estimated spectrallyseparated image vector f. The signal processing circuit Pr converges thesolution through a recursive iterative operation and can calculate theestimated image vector f′ as the final solution.

The expression (6) means to obtain the estimated image vector f′ thatminimizes the sum of the first term and the second term inside the curlybraces on the right-hand side. The first term inside the curly braces inthe above expression represents the amount of deviation between theestimation result Hf and the captured image vector g, or in other words,is a residual term. In the present embodiment, the residual term is thesum of squares of the difference between the acquired captured imagevector g and the matrix Hf obtained by subjecting the spectrallyseparated image vector f in the estimation process to the systemtransformation by the matrix H. Alternatively, the residual term may bean absolute value, a square root of sum of squares, or the like. Thesecond term in the curly braces is a regularization term. The expressionφ(f) in the second term is a constraint in the regularization of thespectrally separated image vector f and is a function that reflectssparse information of the estimated data. The expression acts to smoothor stabilize the estimated data. The regularization term can, forexample, be represented by the discrete cosine transform (DCT) of thespectrally separated image vector f, the wavelet transform, the Fouriertransform, the total variation (TV), or the like. For example, if thetotal variation is used, stable estimated data in which an influence ofnoise of the captured image vector g is suppressed can be acquired. Thesparseness of the object O in the space of each regularization termdiffers depending on the texture of the object O. A regularization termin which the texture of the object O becomes sparser in the space of theregularization term may be selected. Alternatively, a plurality ofregularization terms may be included in an operation. The expression τin the second term is a weighting factor, and as the value of τ isgreater, the amount of the redundant data to be reduced increases, or inother words, the compression rate increases. As the value of τ issmaller, the convergence to the solution is reduced. The weightingfactor τ is set to an appropriate value such that the spectrallyseparated image vector f converges to a certain degree and is notovercompressed.

Here, although an operation example in which the compressed sensingindicated in the expression (6) is illustrated, another technique may beemployed to find a solution. For example, another statistical method,such as a maximum likelihood estimation method and a Bayes estimationmethod, can also be used.

Effects of Fourth Embodiment

The narrow-band coding element includes a plurality oflight-transmitting regions (transmission wavelength ranges T) and aplurality of light-blocking regions (light-blocking wavelength ranges Q)in the wavelength direction. Thus, light that has been coded by thenarrow-band coding element and the spatially modulating coding elementCS and is incident on the image sensor has discrete intensity peaks inthe plurality of wavelength bands corresponding to the transmissionwavelength ranges T. Therefore, in the operation for reconstructing thespectrally separated images F from the light incident on the imagesensor, the number of unknowns to be considered can advantageously bereduced. This is equivalent to that the wavelength resolving power canbe increased. Accordingly, the accuracy of the operation increases, andthus multi-wavelength high-resolution spectrally separated images F canbe obtained.

In addition, light that has passed through a given region A of thespatially modulating coding element is made to be incident only on onecorresponding pixel of the image sensor, and thus light from two or moreregions A is not incident on a single pixel of the image sensor.Accordingly, the operation by the signal processing circuit Pr issimplified.

In addition, when the narrow-band coding element includes thetransmission wavelength ranges T and the light-blocking wavelengthranges Q arrayed periodically in the wavelength direction, thetransmission wavelength ranges T are constantly present at regularintervals. In other words, the transmission wavelength ranges T arepresent in a broader range in the target wavelength band W. Accordingly,spectrally separated images in a greater number of wavelength bands canbe obtained by using a single narrow-band coding element.

In addition, when a plurality of narrow-band coding elements are used,as compared to a case in which a single narrow-band coding element isused, the number of instances of imaging increases, but the number ofthe wavelength bands included in the entire captured image increases.Accordingly, the multi-wavelength spectrally separated images F can beobtained.

When the number of the narrow-band coding elements to be used isincreased, in a case in which, for example, the target wavelength band Wand the number of the wavelength bands stay the same, the number of thetransmission wavelength ranges T in a single narrow-band coding elementcan be reduced. In other words, the range of the light-blockingwavelength ranges Q in a single narrow-band coding element can bebroadened. Accordingly, the number of unknowns in the matrix H isreduced in the operation process for obtaining the spectrally separatedimages by the signal processing circuit Pr, and the calculation issimplified. Thus, the accuracy in reconstructing the spectrallyseparated images can be increased.

In addition, there may be a case in which the number of the transmissionwavelength ranges T in a single narrow-band coding element is limited.In that case, if the target wavelength band is equal, by increasing thenumber of the narrow-band coding elements to be used, the number of thetransmission wavelength ranges T can be increased as a whole. In otherwords, the target wavelength band W can be divided into a larger numberof transmission wavelength ranges T, and thus the wavelength bands canbe narrowed. Thus, an observation in narrower bands becomes possible.

In addition, in the present embodiment, the spatial distribution of theoptical transmittance (spatial distribution of the plurality oflight-transmitting regions and the plurality of light-blocking regions)has wavelength dependence in the spatially modulating coding element.Therefore, light can be coded at the respective wavelengths by thespatially modulating coding element. Therefore, a separate dispersiveelement, such as a prism, does not need to be used, and a typicalimaging lens may be used. Thus, the size of the imaging apparatus can bereduced. In addition, an occurrence of coma aberration arising when adispersive element is used can be suppressed as well, and thus adecrease in the resolution can be suppressed. In addition, in thepresent disclosure in which an image shift for each wavelength by adispersive element is not carried out, the range of saturation of animage arising when intense light is incident on the image sensor islimited, which is advantageous.

What is claimed is:
 1. An imaging apparatus, comprising: a first codingelement that includes regions arrayed two-dimensionally in an opticalpath of light incident from an object; and an image sensor disposed inan optical path of light that has passed through the first codingelement, wherein the regions include a first region and a second region,wherein a wavelength distribution of an optical transmittance of thefirst region has a local maximum in each of a first wavelength band anda second wavelength band that differ from each other, wherein awavelength distribution of an optical transmittance of the second regionhas a local maximum in each of a third wavelength band and a fourthwavelength band that differ from each other, wherein, when thewavelength distribution of the optical transmittance of the first regionis normalized such that the optical transmittance of the first regionhas a maximum value of 1 and a minimum value of 0, the local maxima inthe first wavelength band and the second wavelength band are both noless than 0.5, wherein, when the wavelength distribution of the opticaltransmittance of the second region is normalized such that the opticaltransmittance of the second region has a maximum value of 1 and aminimum value of 0, the local maxima in the third wavelength band andthe fourth wavelength band are both no less than 0.5, wherein at leastone selected from the group of the first wavelength band and the secondwavelength band differs from the third wavelength band and the fourthwavelength band, and wherein, in operation, the image sensor acquires animage in which components of the first wavelength band, the secondwavelength band, the third wavelength band, and the fourth wavelengthband of the light that has passed through the first coding element aresuperimposed on one another.
 2. The imaging apparatus according to claim1, wherein the regions include at least one transparent region.
 3. Theimaging apparatus according to claim 2, wherein the at least onetransparent region comprises a plurality of transparent regions, whereinthe regions include regions whose optical transmittance differs indifferent wavelengths and the plurality of transparent regions, the tworegions are arrayed in an alternating manner in one array direction ofthe regions and another array direction that is perpendicular to the onearray direction.
 4. The imaging apparatus according to claim 1, whereinthe regions are arrayed two-dimensionally in a matrix, wherein a vectorhaving, as its elements, values of transmittance of light in a fifthwavelength band in respective regions belonging to a set of regionsarrayed in a single row or column included in the regions and a vectorhaving, as its elements, values of transmittance of light in the fifthwavelength band in respective regions belonging to a set of regionsarrayed in another row or column included in the regions are independentfrom each other, and wherein a vector having, as its elements, values oftransmittance of light in a sixth wavelength band in respective regionsbelonging to a set of regions arrayed in a single row or column includedin the regions and a vector having, as its elements, values oftransmittance of light in the sixth wavelength band in respectiveregions belonging to a set of regions arrayed in another row or columnincluded in the regions are independent from each other.
 5. The imagingapparatus according to claim 1, further comprising: an optical systemthat is disposed between the object and the first coding element andthat converges the light from the object on a surface of the firstcoding element, wherein the first coding element is disposed on theimage sensor.
 6. The imaging apparatus according to claim 5, wherein theimage sensor includes pixels, and wherein the regions correspond to therespective pixels.
 7. The imaging apparatus according to claim 1,further comprising: an optical system that is disposed between theobject and the first coding element and that converges the light fromthe object on a surface of the image sensor, wherein the first codingelement and the ge sensor are spaced apart from each other.
 8. Theimaging apparatus according to claim 1, further comprising: an opticalsystem that is disposed between the first coding element and the imagesensor and that converges the light from the object that has passedthrough the first coding element on a surface of the image sensor. 9.The imaging apparatus according to claim 1, further comprising: a signalprocessing circuit that, in operation, generates images in respectivewavelength bands of the light that has passed through the first codingelement on the basis of the image acquired by the image sensor and aspatial distribution and a wavelength distribution of an opticaltransmittance of the first coding element.
 10. The imaging apparatusaccording to claim 9, wherein, in operation, the signal processingcircuit generates the images in the respective wavelength bands througha statistical method.
 11. The imaging apparatus according to claim 9,wherein the number of pieces of data in the images in the respectivewavelength bands is greater than the number of pieces of data in theimage acquired by the image sensor.
 12. The imaging apparatus accordingto claim 9, wherein the image sensor includes pixels, and wherein, inoperation, the signal processing circuit generates, as the images in therespective wavelength bands, a vector f′ estimated on the basis of theexpression${f^{\prime} = {\underset{f}{\arg\;\min}\left\{ {{{g - {Hf}}}_{l_{2}} + {{\tau\Phi}(f)}} \right\}}},$wherein φ(f) is a regularization term and τ is a weighting factor, byusing a vector g having, as its elements, signal values of the pixels inthe image acquired by the image sensor and a matrix H determined by thespatial distribution and the wavelength distribution of the opticaltransmittance of the first coding element.
 13. The imaging apparatusaccording to claim 9, wherein, in operation, the signal processingcircuit generates the images in the respective wavelength bands in theform of a moving image.
 14. The imaging apparatus according to claim 1,further comprising: at least one second coding element whose opticaltransmittance is uniform in a spatial direction and that includeslight-transmitting regions and light-blocking regions in the wavelengthdirection, wherein the image sensor is disposed in an optical path oflight that has passed through the first coding element and the at leastone second coding element.
 15. The imaging apparatus according to claim14, wherein, in the at least one second coding element, thelight-transmitting regions have an equal wavelength band width and thelight-blocking regions present between two closest light-transmittingregions in the light-transmitting regions have an equal wavelength bandwidth.
 16. The imaging apparatus according to claim 14, wherein the atleast one second coding element comprises a plurality of second codingelements, and wherein wavelength bands of the light-transmitting regionsin one of the plurality of second coding elements are different fromwavelength bands of the light-transmitting regions in another one of theplurality of second coding elements.
 17. The imaging apparatus accordingto claim 14, further comprising: a signal processing circuit that, inoperation, generates images in respective wavelength bands of the lightthat has passed through the first coding element and the at least onesecond coding element on the basis of the image output by the imagesensor, a spatial distribution and a wavelength distribution of anoptical transmittance of the first coding element, and a wavelengthdistribution of an optical transmittance of the at least one secondcoding element.
 18. The imaging apparatus according to claim 1, whereinthe wavelength distribution of the optical transmittance in each of theregions is a random distribution.
 19. The imaging apparatus according toclaim 1, wherein a spatial distribution of the optical transmittance ofthe first coding element in each of the first wavelength band, thesecond wavelength band, the third wavelength band, and the fourthwavelength band is a random distribution.
 20. A spectroscopic system,comprising: an imaging apparatus that includes a first coding elementthat includes regions arrayed two-dimensionally in an optical path oflight incident from an object, and an image sensor disposed in anoptical path of light that has passed through the first coding element,wherein the regions include a first region and a second region, whereina wavelength distribution of an optical transmittance of the firstregion has a local maximum in each of a first wavelength band and asecond wavelength band that differ from each other, wherein a wavelengthdistribution of an optical transmittance of the second region has alocal maximum in each of a third wavelength band and a fourth wavelengthband that differ from each other, wherein, when the wavelengthdistribution of the optical transmittance of the first region isnormalized such that the optical transmittance of the first region has amaximum value of 1 and a minimum value of 0, the local maxima in thefirst wavelength band and the second wavelength band are both no lessthan 0.5, wherein, when the wavelength distribution of the opticaltransmittance of the second region is normalized such that the opticaltransmittance of the second region has a maximum value of 1 and aminimum value of 0, the local maxima in the third wavelength band andthe fourth wavelength band are both no less than 0.5, wherein at leastone selected from the group of the first wavelength band and the secondwavelength band differs from the third wavelength band and the fourthwavelength band, and wherein, in operation, the image sensor acquires animage in which components of the first wavelength band, the secondwavelength band, the third wavelength band, and the fourth wavelengthband of the light that has passed through the first coding element aresuperimposed on one another; and a signal processing device that, inoperation, generates images in respective wavelength bands of the lightthat has passed through the first coding element on the basis of theimage acquired by the image sensor and a spatial distribution and awavelength distribution of an optical transmittance of the first codingelement.